Making your own server

(Sam Miller) #81

What python version are you using? If it’s version 3 delete the reload(utils) and you should be good.

(Yashar) #82

It is python 2. Here is the version info
Python 2.7.12 |Anaconda 4.2.0 (64-bit)| (default, Jul 2 2016, 17:42:40)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2

(Sam Miller) #83

@yashar56 here are some things to try:

Typing which python from the command line should give you ~/anaconda2/bin/python. If it doesn’t fix your path.

Typing conda install -f bcolz from the command line will reinstall bcolz and check its MD5 hash for corruption.

Lastly, inside a jupyter notebook try running just plain old import bcolz instead of importing utils which is importing bcolz.

(Yashar) #84

I just figured out what was wrong with it. I used sudo when running the and all of the created directories were own by the root user including .jupyter in my home directory.
I just changed the ownership to my own user and this time it works.
chown -R username: ~/.jupyter


I am using a pretty budget setup. AMD fx 6300, with 8 gb ram , and a gtx 1070.
Using a hard drive. I have it set up on a debian 8 headless computer that I purchased just for ML tasks.

(Christopher) #86

AMD doesn’t work well with most ML libraries.

(Jeremy Howard) #87

That’s true for AMD GPUs, but not such a problem for AMD CPUs (although I’d still stick with Intel if possible).

(Christopher) #88

LOL, just noticed he said 1070, I just saw the AMD and thought it was a GPU. I don’t know AMD model numbers at all.


Just in case anybody is on a super tight budget.
My costs are

  • processor(AMD fx 6300) +motherboard (combo) - $110
  • RAM (8gb) - $30
  • ASUS GTX 1070 (got from shenzhen)- $350
  • Case - $30
  • HDD - $20
    Total - ~$550

Although I am not sure how a CPU (and other parts like ram,hdd vs ssd) affects a GPU’s capability in the context of deep learning.
But I am able to run most models on a GPU at sane speeds

(Christopher) #90

Even with a GPU a good CPU makes a significant difference, but you should still get pretty good times with the 1070. A lot better than AWS I’m guessing.


I thought AWS was far more capable than gtx chips. Dont they have >25gb VRAM ?

(Christopher) #92

The AWS instances use a very expensive card ($3,500) compared to a 1070 ($400) but you only get one half of the card (the K80 is basically two cards in one). Doing Part I the 1070 is more than twice the speed of the AWS P2 instance. I haven’t worked on any projects large enough that ram was a deciding factor, but I suspect with the mini batches the ram becomes less of an advantage (within reason).

(Luis Ortega) #93

Hi Sam,

I’m using Python 3.5 and added

from importlib import reload

ahead of the following line

import utils ; reload(utils)

Solved the issue.

(Vishnu Subramanian) #94

What are your recommendations on the below list.

and this

even though i7 has a limit of using upto 64gb ram , why a lot of people are preferring that. I am considering Xeon server which allows scaling till 1.5 tb ram.

Titan X pascal is not available in India ,any idea how can I get it.


gtx1080 ti announced today is cheaper than titan x and faster as well. The xeon’s memory capacity looks impressive.

(Vishnu Subramanian) #96

But it is not available still.

(melissa.fabros) #97

Is this the place to ask about laptops that would be recommended for Deep Learning as well? I’m kind of overdue for an update anyway. Jeremy has his MS Surface Book. I was looking at a Razer Blade 14" which has a GTX 1060 onboard (something about the laptop being VR ready) with 8 GB Memory. There are laptops with a GTX 1080, but that might be too much weight and cost for my needs.

(sai kiran) #98

I’m getting really intimidated by the discussions since I’m not a hardware person. In long term, I do understand that having a personal deep learning machine is beneficial but I’m not able to get my head around it.

Any help?!

(David Gutman) #99

I’d never done anything more technical to computer hardware than replace RAM until this past week - it wasn’t that bad.

Here are the parts of my build with rough price estimates, could definitely be improved. One thing I’m noticing is that I might have been better off with a CPU with more cores - more workers to preprocess data. Would probably get the 6850 instead of 7700 if trying this again (Xeon would be even better if I wanted to make a >2 GPU machine).

Get a CPU with the manufacturer cooler (can be bought without), don’t bother with a third party cooler for now. The first fan I bought was missing pieces, delaying my build, and if it has a backplate it will be the most annoying part of the build (and you need to install before installing your motherboard unless your case has a window to the back of the motherboard).

PCPartPicker part list / Price breakdown by merchant

Type Item Price
CPU Intel Core i7-7700K 4.2GHz Quad-Core Processor $338.89 @ OutletPC
CPU Cooler Cooler Master Hyper 212 EVO 82.9 CFM Sleeve Bearing CPU Cooler $25.88 @ OutletPC
Motherboard Gigabyte GA-Z270X-Gaming 5 ATX LGA1151 Motherboard $180.91 @ Newegg
Memory Corsair Vengeance LPX 32GB (2 x 16GB) DDR4-3000 Memory $254.99 @ Corsair
Storage Samsung 850 Pro Series 512GB 2.5" Solid State Drive $229.99 @ Newegg
Video Card MSI GeForce GTX 1070 8GB Video Card $454.98 @ Newegg
Case Rosewill THOR V2 ATX Full Tower Case $114.72 @ Amazon
Power Supply Corsair 860W 80+ Platinum Certified Fully-Modular ATX Power Supply $154.99 @ Newegg
Prices include shipping, taxes, rebates, and discounts
Total (before mail-in rebates) $1805.35
Mail-in rebates -$50.00
Total $1755.35
Generated by PCPartPicker 2017-03-02 08:05 EST-0500

You also probably want some larger normal hard drives too for extra storage of data you aren’t actively processing.

(Christopher) #100

I don’t see many people having a problem with the i7 memory limit of 64GB.