I have a Keras implementation that looks a lot more like the fast ai lib. forum thread is here and github here.
Right now it gets about 98.5% acc on dogs and cats. I am still looking for the places where fastai (pytorch) and my lib differ to get it up to 99%