Making your own server

What python version are you using? If it’s version 3 delete the reload(utils) and you should be good.

It is python 2. Here is the version info
Python 2.7.12 |Anaconda 4.2.0 (64-bit)| (default, Jul 2 2016, 17:42:40)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2

@yashar56 here are some things to try:

Typing which python from the command line should give you ~/anaconda2/bin/python. If it doesn’t fix your path.

Typing conda install -f bcolz from the command line will reinstall bcolz and check its MD5 hash for corruption.

Lastly, inside a jupyter notebook try running just plain old import bcolz instead of importing utils which is importing bcolz.

I just figured out what was wrong with it. I used sudo when running the install_gpu.sh and all of the created directories were own by the root user including .jupyter in my home directory.
I just changed the ownership to my own user and this time it works.
chown -R username: ~/.jupyter

I am using a pretty budget setup. AMD fx 6300, with 8 gb ram , and a gtx 1070.
Using a hard drive. I have it set up on a debian 8 headless computer that I purchased just for ML tasks.

AMD doesn’t work well with most ML libraries.

That’s true for AMD GPUs, but not such a problem for AMD CPUs (although I’d still stick with Intel if possible).

LOL, just noticed he said 1070, I just saw the AMD and thought it was a GPU. I don’t know AMD model numbers at all.

Just in case anybody is on a super tight budget.
My costs are

  • processor(AMD fx 6300) +motherboard (combo) - $110
  • RAM (8gb) - $30
  • ASUS GTX 1070 (got from shenzhen)- $350
  • Case - $30
  • HDD - $20
    Total - ~$550

Although I am not sure how a CPU (and other parts like ram,hdd vs ssd) affects a GPU’s capability in the context of deep learning.
But I am able to run most models on a GPU at sane speeds

3 Likes

Even with a GPU a good CPU makes a significant difference, but you should still get pretty good times with the 1070. A lot better than AWS I’m guessing.

I thought AWS was far more capable than gtx chips. Dont they have >25gb VRAM ?

The AWS instances use a very expensive card ($3,500) compared to a 1070 ($400) but you only get one half of the card (the K80 is basically two cards in one). Doing Part I the 1070 is more than twice the speed of the AWS P2 instance. I haven’t worked on any projects large enough that ram was a deciding factor, but I suspect with the mini batches the ram becomes less of an advantage (within reason).

3 Likes

Hi Sam,

I’m using Python 3.5 and added

from importlib import reload

ahead of the following line

import utils ; reload(utils)

Solved the issue.

What are your recommendations on the below list.
https://in.pcpartpicker.com/list/DNjdqk

and this

even though i7 has a limit of using upto 64gb ram , why a lot of people are preferring that. I am considering Xeon server which allows scaling till 1.5 tb ram.

Titan X pascal is not available in India ,any idea how can I get it.

gtx1080 ti announced today is cheaper than titan x and faster as well. The xeon’s memory capacity looks impressive.

But it is not available still.

Is this the place to ask about laptops that would be recommended for Deep Learning as well? I’m kind of overdue for an update anyway. Jeremy has his MS Surface Book. I was looking at a Razer Blade 14" which has a GTX 1060 onboard (something about the laptop being VR ready) with 8 GB Memory. There are laptops with a GTX 1080, but that might be too much weight and cost for my needs.

I’m getting really intimidated by the discussions since I’m not a hardware person. In long term, I do understand that having a personal deep learning machine is beneficial but I’m not able to get my head around it.

Any help?!

I’d never done anything more technical to computer hardware than replace RAM until this past week - it wasn’t that bad.

Here are the parts of my build with rough price estimates, could definitely be improved. One thing I’m noticing is that I might have been better off with a CPU with more cores - more workers to preprocess data. Would probably get the 6850 instead of 7700 if trying this again (Xeon would be even better if I wanted to make a >2 GPU machine).

Get a CPU with the manufacturer cooler (can be bought without), don’t bother with a third party cooler for now. The first fan I bought was missing pieces, delaying my build, and if it has a backplate it will be the most annoying part of the build (and you need to install before installing your motherboard unless your case has a window to the back of the motherboard).

PCPartPicker part list / Price breakdown by merchant

Type Item Price
CPU Intel Core i7-7700K 4.2GHz Quad-Core Processor $338.89 @ OutletPC
CPU Cooler Cooler Master Hyper 212 EVO 82.9 CFM Sleeve Bearing CPU Cooler $25.88 @ OutletPC
Motherboard Gigabyte GA-Z270X-Gaming 5 ATX LGA1151 Motherboard $180.91 @ Newegg
Memory Corsair Vengeance LPX 32GB (2 x 16GB) DDR4-3000 Memory $254.99 @ Corsair
Storage Samsung 850 Pro Series 512GB 2.5" Solid State Drive $229.99 @ Newegg
Video Card MSI GeForce GTX 1070 8GB Video Card $454.98 @ Newegg
Case Rosewill THOR V2 ATX Full Tower Case $114.72 @ Amazon
Power Supply Corsair 860W 80+ Platinum Certified Fully-Modular ATX Power Supply $154.99 @ Newegg
Prices include shipping, taxes, rebates, and discounts
Total (before mail-in rebates) $1805.35
Mail-in rebates -$50.00
Total $1755.35
Generated by PCPartPicker 2017-03-02 08:05 EST-0500

You also probably want some larger normal hard drives too for extra storage of data you aren’t actively processing.

1 Like

I don’t see many people having a problem with the i7 memory limit of 64GB.

1 Like