[ EDIT - 05/12/2020 ] I cc @sgugger because I did not receive an answer about my question on the increasing size of the pkl file when the size batch is larger. However, I think that it is a significant problem,
Hello. I am using this thread for another issue regarding learn.export()
(at least, one question I have).
The size of the pkl file created by learn.export()
depends on the batch size (at least in my test, it depends on large batch size).
I do not understand why. Any idea?
My notebook is online in github and the following table shows my results (sizes of pkl files by batch size from 8 to 512). The smallest pkl file (bs from 8 to 128) has a size of 52M and the biggest (bs of 512) of 272M.