AWS EMR issues

[DallanQ]

I had some problems running on AWS EMR with the default mrjob.conf. In case anyone else is running into similar issues, I found that I needed to make two minor changes to mrjob.conf: change python2.7 and python2.7-pip to python2 and python2-pip, and change boto to boto3.

- sudo yum install -y python2 python2-pip python2-devel gcc-c++
- sudo pip-2.7 install boto3 mrjob simplejson warc

I really like this project - much simpler to use than cc-pyspark. I hope it continues to work past July 15 when boto3 no longer supports python 2.7.

[sebastian-nagel]

Thanks, @DallanQ! I'll update the mrjob.conf but should also get a Python3 version ready (see #11), just in case.

[DallanQ]

Thank you @sebastian-nagel! This really is a terrific project. It makes it so easy to process the CommonCrawl.