Is anyone having problems with the files that the kaggle cli is downloading to AWS (M4.large instance)?
I fixed the file structure in the create_folders.py file and now i’m getting an error that says the downloaded files aren’t zip files
I tried to unzip them manually but I’m getting an error as well where unzip doesn’t work test.zip
The error message suggests that the files need to be unzipped in a particular order? I’m getting the same error when apply unzip to train.zip as well.
Not sure why. Can you try ls -l train.zip and see if you get something equivalent of ~500 MB? It worked for me in the t2.micro instance.
To cut costs, I would recommend doing any non deep learning task on a micro instance. Then we can switch the same instance to a p2 and run DL scripts. That way all the data stored under a micro instance can be reused when using the p2 instance. At least thats what I think will work … correct me if I am wrong.
It worked, I had to go to kaggle page with data with browser and I tried to download link from there and then it asked for approval of competition rules again. And then I ran kg download and then the files worked. The download starts with a connection from ww.kaggle.com, but the valid files seem to come from kaggle2.blob.core.windows.net
I would like to know how to submit to Kaggle.
The sample_submission.csv file has 2 column headers id and label. I am supposing id refers to the image id. Does label refer to the probability of prediction or the category of the image?
Also, how do I generate a csv file from the output of vgg.predict?
I am new to Kaggle. So please excuse me if this is something very basic. I am also trying to find out myself.
@swathi.ssunder Looking at sample_submission.csv is a good way of seeing what the data should look like (if the values are probabilities or categories), but make sure that you are using sample_submission.csv for the right competition. Dogs-vs-cats and Dogs-vs-cats-redux-kernels-edition have different required submission formats, and part of the homework is figuring out how to process and format the output correctly.
As for generating a csv (from a numpy array), can you tell me what you’ve tried searching for so far, or what methods you’ve already tried? I can give you a hint if you are stuck.
I have a Kaggle set up question. I tried pip install kaggle-cli in Cygwin at Windows 7, but I am not sure it was successful. I tried pip install python-lxml, but I am not sure it worked either. What shall I do?
kicho@PC08EYW5 ~
$ pip install kaggle-cli
The directory '/home/kicho/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/kicho/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting kaggle-cli
Requirement already satisfied (use --upgrade to upgrade): cliff in /usr/lib/python2.7/site-packages (from kaggle-cli)
Requirement already satisfied (use --upgrade to upgrade): MechanicalSoup in /usr/lib/python2.7/site-packages (from kaggle-cli)
Collecting lxml (from kaggle-cli)
Downloading lxml-3.6.4.tar.gz (3.7MB)
100% |████████████████████████████████| 3.7MB 195kB/s
Collecting cssselect (from kaggle-cli)
Downloading cssselect-1.0.0-py2.py3-none-any.whl
Collecting configparser (from kaggle-cli)
Requirement already satisfied (use --upgrade to upgrade): cmd2>=0.6.7 in /usr/lib/python2.7/site-packages (from cliff->kaggle-cli)
Requirement already satisfied (use --upgrade to upgrade): PrettyTable<0.8,>=0.7 in /usr/lib/python2.7/site-packages (from cliff->kaggle-cli)
Requirement already satisfied (use --upgrade to upgrade): unicodecsv>=0.8.0; python_version < "3.0" in /usr/lib/python2.7/site-packages (from cliff->kaggle-cli)
Requirement already satisfied (use --upgrade to upgrade): pbr>=1.6 in /usr/lib/python2.7/site-packages (from cliff->kaggle-cli)
Requirement already satisfied (use --upgrade to upgrade): stevedore>=1.16.0 in /usr/lib/python2.7/site-packages (from cliff->kaggle-cli)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9.0 in /usr/lib/python2.7/site-packages (from cliff->kaggle-cli)
Requirement already satisfied (use --upgrade to upgrade): PyYAML>=3.1.0 in /usr/lib/python2.7/site-packages (from cliff->kaggle-cli)
Requirement already satisfied (use --upgrade to upgrade): pyparsing>=2.0.1 in /usr/lib/python2.7/site-packages (from cliff->kaggle-cli)
Requirement already satisfied (use --upgrade to upgrade): beautifulsoup4 in /usr/lib/python2.7/site-packages (from MechanicalSoup->kaggle-cli)
Requirement already satisfied (use --upgrade to upgrade): requests>=2.0 in /usr/lib/python2.7/site-packages (from MechanicalSoup->kaggle-cli)
Installing collected packages: lxml, cssselect, configparser, kaggle-cli
Running setup.py install for lxml ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-xCaOTi/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-QWptuE-record/install-record.txt --single-version-externally-managed --compile:
Building lxml version 3.6.4.
Building without Cython.
ERROR: /bin/sh: xslt-config: command not found
** make sure the development packages of libxml2 and libxslt are installed **
Using build configuration of libxslt
running install
running build
running build_py
creating build
creating build/lib.cygwin-2.6.0-x86_64-2.7
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/builder.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/cssselect.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/doctestcompare.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/ElementInclude.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/pyclasslookup.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/sax.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/usedoctest.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/_elementpath.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/__init__.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/__init__.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/builder.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/clean.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/defs.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/diff.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/ElementSoup.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/formfill.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/html5parser.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/soupparser.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/usedoctest.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/_diffcommand.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/_html5builder.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/_setmixin.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/__init__.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron
copying src/lxml/isoschematron/__init__.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron
copying src/lxml/lxml.etree.h -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/lxml.etree_api.h -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/includes/c14n.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/config.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/dtdvalid.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/etreepublic.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/htmlparser.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/relaxng.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/schematron.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/tree.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/uri.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/xinclude.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/xmlerror.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/xmlparser.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/xmlschema.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/xpath.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/xslt.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/etree_defs.h -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/lxml-version.h -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/rng
copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/rng
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
running build_ext
building 'lxml.etree' extension
creating build/temp.cygwin-2.6.0-x86_64-2.7
creating build/temp.cygwin-2.6.0-x86_64-2.7/src
creating build/temp.cygwin-2.6.0-x86_64-2.7/src/lxml
gcc -fno-strict-aliasing -ggdb -O2 -pipe -Wimplicit-function-declaration -fdebug-prefix-map=/usr/src/ports/python/python-2.7.12-1.x86_64/build=/usr/src/debug/python-2.7.12-1 -fdebug-prefix-map=/usr/src/ports/python/python-2.7.12-1.x86_64/src/Python-2.7.12=/usr/src/debug/python-2.7.12-1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Isrc/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.cygwin-2.6.0-x86_64-2.7/src/lxml/lxml.etree.o -w
unable to execute 'gcc': No such file or directory
Compile failed: command 'gcc' failed with exit status 1
creating tmp
cc -I/usr/include/libxml2 -c /tmp/xmlXPathInitAWWi_U.c -o tmp/xmlXPathInitAWWi_U.o
unable to execute 'cc': No such file or directory
*********************************************************************************
Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
*********************************************************************************
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-xCaOTi/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-QWptuE-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-xCaOTi/lxml/
kicho@PC08EYW5 ~
$ pip install python-lxml
The directory '/home/kicho/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/kicho/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting python-lxml
Could not find a version that satisfies the requirement python-lxml (from versions: )
No matching distribution found for python-lxml
@analytics20132333 it would be ‘pip install lxml’, not ‘pip install python-lxml’. Having said that, there’s really no reason to install kaggle cli on Windows - you can just use your browser to submit to Kaggle on Windows.
But I’m guessing this isn’t actually what you want - assuming that you’re doing your GPU training on Linux (i.e. an AWS instance), then you need to be installing kaggle CLI onto your GPU server, not into Cygwin. The purpose of Kaggle CLI is that it’s hard to navigate the Kaggle website and make a submission via your Linux server, so it’s generally easier to to submit through a command line interface (CLI). So you would type ‘pip install kaggle-cli’ into your bash shell on your server.
The get_batches function presumes the passed-in path will contain subdirectories, so you’ll want to create a subdirectory within your test folder (you can name it anything you want), and move all of the images into that subdirectory.
If you’re wondering how to get the filenames for the batched images, there is a .filenames method you can call…
Hi @jeremy! I think it makes sense. I was simply following the assignment instruction. I was at Cygwin level, but I tried again on my AWS instance after reading your comment. I got the same error after trying pip install lxml. Is there a way to make the command line interface installation possible on an AWS instance? Or should I use a browser in Windows?
kicho@PC08EYW5 ~
$ pip install lxml
The directory '/home/kicho/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/kicho/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting lxml
Downloading lxml-3.6.4.tar.gz (3.7MB)
100% |████████████████████████████████| 3.7MB 276kB/s
Installing collected packages: lxml
Running setup.py install for lxml ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-AeWcP9/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-4tvJdQ-record/install-record.txt --single-version-externally-managed --compile:
Building lxml version 3.6.4.
Building without Cython.
ERROR: /bin/sh: xslt-config: command not found
** make sure the development packages of libxml2 and libxslt are installed **
Using build configuration of libxslt
running install
running build
running build_py
creating build
creating build/lib.cygwin-2.6.0-x86_64-2.7
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/builder.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/cssselect.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/doctestcompare.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/ElementInclude.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/pyclasslookup.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/sax.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/usedoctest.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/_elementpath.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/__init__.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/__init__.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/builder.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/clean.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/defs.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/diff.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/ElementSoup.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/formfill.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/html5parser.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/soupparser.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/usedoctest.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/_diffcommand.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/_html5builder.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/_setmixin.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
copying src/lxml/html/__init__.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/html
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron
copying src/lxml/isoschematron/__init__.py -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron
copying src/lxml/lxml.etree.h -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/lxml.etree_api.h -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml
copying src/lxml/includes/c14n.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/config.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/dtdvalid.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/etreepublic.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/htmlparser.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/relaxng.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/schematron.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/tree.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/uri.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/xinclude.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/xmlerror.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/xmlparser.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/xmlschema.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/xpath.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/xslt.pxd -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/etree_defs.h -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
copying src/lxml/includes/lxml-version.h -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/includes
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/rng
copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/rng
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl
creating build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.cygwin-2.6.0-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
running build_ext
building 'lxml.etree' extension
creating build/temp.cygwin-2.6.0-x86_64-2.7
creating build/temp.cygwin-2.6.0-x86_64-2.7/src
creating build/temp.cygwin-2.6.0-x86_64-2.7/src/lxml
gcc -fno-strict-aliasing -ggdb -O2 -pipe -Wimplicit-function-declaration -fdebug-prefix-map=/usr/src/ports/python/python-2.7.12-1.x86_64/build=/usr/src/debug/python-2.7.12-1 -fdebug-prefix-map=/usr/src/ports/python/python-2.7.12-1.x86_64/src/Python-2.7.12=/usr/src/debug/python-2.7.12-1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Isrc/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.cygwin-2.6.0-x86_64-2.7/src/lxml/lxml.etree.o -w
unable to execute 'gcc': No such file or directory
Compile failed: command 'gcc' failed with exit status 1
creating tmp
cc -I/usr/include/libxml2 -c /tmp/xmlXPathInitd1Hw8A.c -o tmp/xmlXPathInitd1Hw8A.o
unable to execute 'cc': No such file or directory
*********************************************************************************
Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
*********************************************************************************
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-AeWcP9/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-4tvJdQ-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-AeWcP9/lxml/
As it mentions in the error ‘make sure the development packages of libxml2 and libxslt are installed’. They should already be part of anaconda - are you using our script to setup your AWS instance? It looks like you are either not using it, or are not connected to your instance, since your prompt above shows ‘kicho@PC08EYW5 ~’, but on our AWS setup your username would be ‘ubuntu’, not ‘kicho’.
On our AWS setup, you should not need to install any prerequisites in order to install kaggle-cli - just ‘pip install kaggle-cli’ should be it.
I was getting an error
ubuntu@ip-10-0-0-5:~/nbs$ kg download
Starting new HTTPS connection (1): www.kaggle.com
’NoneType’ object has no attribute ‘find_all’
There was a typo in the competition name. I was using dogs-vs-cats-redux-kernel-edition instead of dogs-vs-cats-redux-kernels-edition. That fixed it.
@rachel - Thank you for pointing me to the evaluation page, where there is info about the submission s.
My question was about how to actually run the trained model on the test data? Was I right in trying to execute vgg.predict on the test data?
When I actually did this, the result was a tuple, which I converted to a numpy array through numpy.asarray and then wrote to a csv file.
I think there is something wrong in this.
@swathi.ssunder you’re on the right track, but you’ll need to spend some time looking at the data structure that’s returned from predict(), and figuring out how to modify it to be the correct structure to submit to kaggle. You’ll need to write some code to make these changes.
Give it your best shot, and if you get stuck, let us know exactly what you’ve tried, what you’ve found so far, and what it is that you need to do next and don’t know how to do.
I used SciKit Learn’s StratifiedKFold and generating symlinks in python to dynamically generate the training and validation sets (and reset them, if necessary). I hope that’s ok.