Meta: Getting unstuck in deep learning

Deep Learning
1

As a beginner in Deep Learning, I’m running into a recurring pattern of getting stuck.

  1. Read basic information around an area e.g. get to grips with what an LSTM is.
  2. Follow some tutorials. Be able to get them running fine, maybe tweaking slightly on a different dataset.
  3. Try and implement it on my own project.
  4. Get stuck in a thousand tiny details, spend forever googling and not make progress I am happy with.

I’m guessing this is pretty common but I still think I could take some steps to alleviate the issue and make faster progress.

What do people who have got over this ‘hump’ stage recommend? Here are some ideas I had:

  • Mentorship. Find someone more experience for regular check-ins and question asking.
  • Get better at asking questions online. Maybe I should be post a lot more on these forums. The loop of posting and waiting feels a little slow though.
  • Go foundational. Study some more maths, Python, and neural net foundations so I have more of a framework to place my knowledge on.
  • Go slower. Rather than going deeper to help questions stick, maybe I should be going a lot slower through materials. What if I didn’t go to the next line in the textbook until I fully understood it?
  • Don’t change anything. Maybe this is just what it is like? Learning can be hard and that’s okay!
3 Likes
2

Hey!

I’ve been there SO many times and it is really frustrating! I guess with all things learning, it’s never going to be easy breezy, but totally agree on trying to work smarter rather than harder.

What you’ve said sounds pretty much how I do things (especially point 3) so I think you’ve definitely go the right fundamental approach (at least in my opinion). It’s a bit difficult to say anything too specific - I’m not sure what’s your favored way of learning, or your is project, but I’ll give it a go!

Mentors

I’ve never really had one so can’t comment much on them

Asking questions online:

This is something I wish I had done more of when I was first starting out, not always a guaranteed answer but if you’re stuck and can’t find anything its always worth a post.

Go foundational:

This is a tricky one to answer because its not always easy to tell. Doing more foundational stuff can feel good because you’re learning stuff and knowing more things must be good…well (and I can only go from my experience) this feeling is a bit misleading.

I’ve often found that having gone away and done more learning about the fundamentals of the problem it hasn’t really helped that much in my specific case. Spending time experimenting and troubleshooting the problem, has pretty much always been the most efficient use of my time - mainly because, after awhile, it shows you what the ‘actual’ specific problem (which you can then go deep on if you need to).

Slower:

Maybe. Sometimes I’ve found pushing on even if I don’t totally understand everything helps because the new material might help explain/ make me think about the problem differently . But at the same time if you really don’t get something going back and re-watching a concept is also good - even better (for me at least) is to try and find an explanation of that particular idea from somewhere else. Watching the same thing over and over again doesn’t do if for me, the more ways of explaining it the better!

Don’t change anything

In essence yeah, but I promise it does gets easier! Partly, you just get better at searching for answers but also because even if you don’t think you are understanding anything you probably are, it just takes time to click into place.

TLDR;

  • Ask questions on the forum
  • Just trying the problem > reading around the problem
  • Don’t feel afraid to keep going if you don’t understand everything
  • Don’t feel afraid to go back, if you don’t get something
  • Don’t give up

Above all else, stay positive, take brakes and keep going!

You might like this video, the video title says it’s about galaxy photography but its actually about learning new skills, I often think about it when I’m stuck (so I think about this video a lot)!

Hope that helps

What’s your project about?

3 Likes
3

Thanks for the reply! The foundational and slower responses were particularly interesting.

I’m building https://typeperf.com/ so I want to build a model that generates fake code to practice typing on, then go on to build a predictive model for typos so that I can generate the most efficient text to practice on. I have my galaxy to aim for!

4

I think 90% of the people on this forum share both your views.

I also have spent a lot of my time feeling stuck, and am always trying to go faster.

Recently I have been reflecting on finding my most productive learning style, I have found the following method helpful.

Pseudo understanding

To know how much I really understand a subject.

I take a blank sheet of paper and pencil and write, scribble, draw, illustrate everything I now about the subject.

Often I am surprised how little/much I know about a given subject I am about to work on.

In reality for me videos, tutorials and just reading, are only useful for showing me the specific syntax of a command or how to use a library correctly.

When I do the blank sheet of paper test after completing these activities I find I haven’t learn’t much and am really only remembering what I have recently seen or read.

What really works for me is to visualize and create my own solution of what I am trying to achieve, in my mind and on a blank piece of paper with a pencil, this ends up being a poor drawing :frowning_face: some pseudo code and other hieroglyphics. I do this with as much detail as i feel is necessary to complete/uderstand the solution.

I have found that sometimes my solution is better than the tutorial, other times the tutorial is better, however the real benefit is that, now I just write some bits of code and use some bits of code from the tutorial and get a better understanding. When I do the blank paper and pen test weeks later I am able to remember most of the details. This just doesn’t happen when I blindly follow/copy any other materials.

Also once I have done this process, although sometimes I just want to start coding, I have found my coding output has quadrupled even with slow typing.

I spend less time being stuck, and being stuck is far more enjoyable because often I can see the floor in my thinking and other times I literally diverge from the tutorial or solution and write my own functions in parts of the tuttorial.

I have found virtually a 100% correlation with getting stuck and how little I truly understand a project/subject.

I think this method was used by https://en.wikipedia.org/wiki/Richard_Feynman

Cheers mrfabulous1 :smiley: :smiley:

3 Likes
5

I came across the Feynman Technique in Scott Young’s Ultralearning. I’ll definitely be trying to apply it more now

6

Hi mulholio hope all is well!
I use it before I start to write any new function, I often don’t go through thte whole process.

For example I neeeded two write a function to multiply two n dimensional vectors together.

I got a blank sheet of paper out scribbled, sketched and psuedo coded my best effort of what this function should look like.

def n_x_vectors(v1,v2):
    get v1
    get v2
    v3 = v1 x v2
   return v3

I tried to remember the maths but could only remember pythagorous and soh cah toa for working out angles.

The above process is rapid, seconds or minutes depending on my understanding.

Now I search for just the bit of the algorithm I don’t understand NEVER the whole solution and refine my solution. I iterate this process until the function is complete.

When it is working, I then look at other solutions to see if I could have done it better.

The other day I wrote a function and used about 50 lines of code, aftter I finished the process and wrote the function. I looked on stack.overflow.com and found someone had written it in about 5 lines.

I just laughed, as now I really understood the subect matter.

I try and make programming fun, as it’s tricky enough without me getting fustrated.

Cheers mrfabulous1 :smiley: :smiley:

2 Likes
7

Oops! I hadn’t quite finished the post… :grinning: :grinning:

1 Like