Skip to content

Latest commit

 

History

History
48 lines (40 loc) · 1.85 KB

README.mkd

File metadata and controls

48 lines (40 loc) · 1.85 KB
Raw

SCRAPY WORKSHOP SETUP

You need ipython and scrapy, python mysql bindings mysqldb and mysql server mysql or mariadb (to run a mysql server and write some data into a database table from within our scraper).

Should be rather easy to install those systemwide or with pip,

pip install ipython Scrapy MySQL-python

To install mysql-server on ubuntu for example,

sudo apt-get install mysql-server

SKLEARN WORKSHOP SETUP

If you have scikit-learn, matplotlib and ipython installed you can just run ipython notebook --pylab. Otherwise there are 3 different ways of installing the tools below.

  • If you're on Windows or Mac OSX I recommend setting up a VM.
  • If you're running a Linux distro with package manager I recommend either install the depencies system wide via the package manager or setting up a virtual python environment and installing the tools via pip.

Set Up a VM

  • install Virtualbox from https://www.virtualbox.org/wiki/Downloads
  • download a ubuntu virtualbox image from http://virtualboxes.org/images/ubuntu/, 12.04 or newer recommended
  • add the appliance to virtualbox, click Machine -> Add, then navigate to the *.vbox file
  • once added, fire up the vm, log in with username/password provided on virtualboxes.org
  • in a terminal, run sudo apt-get update
  • followed by sudo apt-get install ipython python-matplotlib python-sklearn

Debian / Ubuntu

Run

sudo apt-get install ipython python-matplotlib python-sklearn

In a virtual environment

If inside a virtual environment run

pip install numpy scipy matplotlib ipython scikit-learn

you may need to install some additional system level dependencies if this command fails.

Start the ipython notebook server

Run ipython notebook --pylab from within this repository and open a new notebook.