@radek doing this course from scratch (without using the fastai lib) will be much harder than just avoiding utils.py from last year’s part 1. utils.py was just a few little convenience functions, but fastai contains the result of quite a bit of new research. You might be better off updating your goal to something involving helping to document or improve the fastai lib, rather than avoid it entirely.
Rewriting bits of it from scratch will certainly help you learn the material - just want you to not be too disappointed if you can’t do all of if from scratch!