How do I build my own activation function

I have an idea for an activation function that I want to experiment with it has to work with a range of libraries because this is how I am learning right now (Of course these are for Python libraries).

The best contact for this would likely be @Diganta, who came up with the Mish activation function that is being used in many research papers, and probably even industry.

1 Like

the latter part of this thread should get you going.

Thanks for the tag @ilovescience. Happy to help. My package Echo would be a good step to start experimenting with several activation functions and build your own one in PyTorch/ Keras. Let me know if I can be of any help.