Sure, I’ve been using Floydhub. GPU’s are the same (K80 and V100s)…however Salamander pricing is much cheaper (esp for V100…about 2/3 cheaper as Floyd is $4.20/hr vs $1.61 at Salamander).
Also, I noted that Salamander shows the Cpu and GPU on the server choices…vs Floyd just shows GPU and doesn’t mention CPU’s.
And their base CPU only at Floyd is a Xeon x2 core. Vs. 8x CPU shown with the V100 on Salamander.
As to how that plays out - when I uploaded and unzipped the NIH malaria dataset…it took literally 5+ minutes to unzip on Floyd. (and I did it 2x both on K80 and V100), no difference.
I thought maybe the images were super compressed or something and that’s why it was so slow…but on Salamander, it unzipped in roughly 30 seconds! (also done 2x). Huge difference.
Training overall seemed faster as well with the Salamander V100, I’m guessing b/c the cpu’s could keep things moving to the GPU faster. (For reference, Floyd kept blowing up with EOF error that was unexplainable and I wasted 4+ hours with it… no problem on Salamander. I’m now suspecting it was a timing issue due to the slower CPU on Floyd not keeping up).
The main things that would be great to see in the future on Salamander is:
1 - an auto-shutoff after idle, as I often kick off longer runs at night and just let it go into the night and know it will auto stop.
2 - nice to have - if it’s possible to show GPU activity and most importantly, consumed memory, as I use that as rough guide to adusting batch sizes, which is vital esp when doing segmentation work.
Anyway, Salamander is a nice platform and I’m going to switch over completely after I use up all my pre-purchased floyd credits.
Hey everyone, does anyone know how to get salamander to run Jupyter notebooks in the background? Basically, if I call learn.fit() I want to be able to walk away from my computer if it’s going to take a long time. I did some cursory googling, and it seems like most people recommend tmux or nohup, but they seem pretty complicated and I don’t really want to have to learn them. Is there an easy way to do this?
Ok awesome! I asked because earlier today I turned off my computer with the server running and came back to an error message that made it seem like it had timed out. Unfortunately I didn’t save a copy of it, but if it comes up again I’ll come back here.
Ah yes you can’t turn your computer off - notebook always requires a browser to keep running. That’s not a salamander issue - that’s just how notebooks work.
I have been unable to open any notebooks. I was getting a 500 Internal Server Error. I have done a git pull without incident and a conda update without incident.
I tried your suggestion of conda update --all. Now the Jupyter notebook won’t open at all. I just get an infinite “Waiting for Jupyter to start…”
@Mark_F unfortunately there’s been recent updates to tornado, a package that jupyter depends on, which are causing problems with jupyter. Try reverting your version to an older one:
And it never actually opens the notebook. So I can’t actually open the terminal. Is there some other way to get into the terminal? I don’t really know what I’m doing with ssh but I tried generating an SSH key pair and it also did not work, just spinning dots, followed by an error:
Should I just create a new server? Not preferred option because I do have some things saved, but not the end of the world, either.
Can someone help me with the Salamander free credits?
I have github education pack, but in AWS educate I only got $30 in credits as my institution is not registered with Amazon. Also, I can’t seem to find the coupon code. My Educate account was activated automatically without me requiring to enter promo code. What should I do?
Is there any workaround to get credits worth $100?
I am still unable to open Jupyter in my server. I cannot revert Tornado as Jeremy has suggested because I can’t even get to the point where I can open a terminal.
Should I just blow up my server and start a new one? Will a new server work with the newest git pull and fastai updates? Is anyone else having this problem?
@jeremy I’m really sorry to bother you with this, but I’m not sure where to go from here. Any thoughts?
@Mark_F can you use ssh to login to your box (the ssh details are on the dashboard)? If you can’t even ssh, then yes you’ll need to create a new server. The newest servers should absolutely work fine - we just built a new image a few days ago and tested it.
Hey,
I have Github Education and tried to enter my AWS code on salamander, but it keeps giving me “invalid promotion code”, although I didn’t use the code and it was not expired. How to get the free credit on salamander using Github Education?
update - tried on a second (new) server, same exact issue. So total of 3x.
My workaround - upload zip file to my own server, use wget to access it and download…that works.
But really need to be able to upload datasets directly!
I’m unable to upload to the server today and have reproed this issue 2x.
It let’s me select the file, shows it with the name in an editable text box and the upload/cancel buttons…and then you can’t do anything after that. Hitting upload doesn’t respond, cancel doesn’t respond, etc. It just hangs.
I’ve reproed this 2x today - shut down and restarted the server, both times exact same issue.
Since I can’t upload the dataset of course I can’t do any work so unfortunately back to Floyd for today.
@LessW2020 works for me! Try creating a different directory, or using a different extension on the file, perhaps? Does a smaller file work? Shoot me a PM if you have more details and I’ll try to help resolve it.
Hi, I’ve signed up for a Salamander instance and tried to upload SSH keys, but after selecting the .pub file the window doesn’t appear to do anything. I don’t see an error, nor do I see any visual confirmation that my key is uploaded. When I try to ssh ubuntu@xxx.yyy.zzz.aaa, it fails with a permission error.
I have also tried to use the ‘Generate Keys’ option, but that hangs and does nothing.
First of all, Salamander is a great service, it’s been very helpful.Thank you.
With that being said my servers have been randomly turning off and I can no longer trust Salamander with my training as it keeps getting interrupted and work goes to waste let alone the money.
Has this been a recent issue? Do you know why this happens?
