Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bulletproof #139

Open
wants to merge 171 commits into
base: master
Choose a base branch
from
Open

bulletproof #139

wants to merge 171 commits into from

Commits on Apr 12, 2018

  1. initial loader

    @eads
    eads committed Apr 12, 2018
    Configuration menu
    Copy the full SHA
    9d79555 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2018

  1. initial commit

    @eads
    eads committed Apr 24, 2018
    Configuration menu
    Copy the full SHA
    5323d84 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2018

  1. WIP makefile + benchmarks

    @eads
    eads committed Jun 11, 2018
    Configuration menu
    Copy the full SHA
    3449dc5 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2018

  1. add some git keeps

    @eads
    eads committed Jun 13, 2018
    Configuration menu
    Copy the full SHA
    0fb037a View commit details
    Browse the repository at this point in the history

  2. improve makefile and add camera data loader

    @eads
    eads committed Jun 13, 2018
    Configuration menu
    Copy the full SHA
    13a2933 View commit details
    Browse the repository at this point in the history

  3. add sql definitions for parking and cameras, though the schema is ide… 

    …ntical
    @eads
    eads committed Jun 13, 2018
    Configuration menu
    Copy the full SHA
    d674643 View commit details
    Browse the repository at this point in the history

  4. add csv cleaning processor

    @eads
    eads committed Jun 13, 2018
    Configuration menu
    Copy the full SHA
    f94b7e6 View commit details
    Browse the repository at this point in the history

  5. add dev env var file

    @eads
    eads committed Jun 13, 2018
    Configuration menu
    Copy the full SHA
    58490be View commit details
    Browse the repository at this point in the history

  6. remove duplicate recipe and consolidate aliases

    @eads
    eads committed Jun 13, 2018
    Configuration menu
    Copy the full SHA
    4a04340 View commit details
    Browse the repository at this point in the history

  7. update readme to account for consolidated aliases and some broken parts

    @eads
    eads committed Jun 13, 2018
    Configuration menu
    Copy the full SHA
    9759acc View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2018

  1. add clean commands

    @eads
    eads committed Jul 6, 2018
    Configuration menu
    Copy the full SHA
    1c690cb View commit details
    Browse the repository at this point in the history

  2. move up download command

    @eads
    eads committed Jul 6, 2018
    Configuration menu
    Copy the full SHA
    4816597 View commit details
    Browse the repository at this point in the history

  3. fix circular reference

    @eads
    eads committed Jul 6, 2018
    Configuration menu
    Copy the full SHA
    d6a6e3e View commit details
    Browse the repository at this point in the history

  4. fix typo in download rule

    @eads
    eads committed Jul 6, 2018
    Configuration menu
    Copy the full SHA
    f2eb5d7 View commit details
    Browse the repository at this point in the history

  5. download command for cameras

    @eads
    eads committed Jul 6, 2018
    Configuration menu
    Copy the full SHA
    a47a428 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2018

  1. change error file from .csv to .txt, closes #8

    kat-alo committed Jul 9, 2018
    Configuration menu
    Copy the full SHA
    d6cb835 View commit details
    Browse the repository at this point in the history

  2. change psql table name from 'tickets' to 'parking', closes #10

    kat-alo committed Jul 9, 2018
    Configuration menu
    Copy the full SHA
    774d81f View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2018

  1. add skeletal file

    @eads
    eads committed Jul 11, 2018
    Configuration menu
    Copy the full SHA
    a032a23 View commit details
    Browse the repository at this point in the history

  2. factor out cleaning functions

    @eads
    eads committed Jul 11, 2018
    Configuration menu
    Copy the full SHA
    25ed78c View commit details
    Browse the repository at this point in the history

  3. simple offset-based geocoder script, plus related schema updates

    @eads
    eads committed Jul 11, 2018
    Configuration menu
    Copy the full SHA
    83f0d84 View commit details
    Browse the repository at this point in the history

  4. Add geocodes table

    @eads
    eads committed Jul 11, 2018
    Configuration menu
    Copy the full SHA
    9a07337 View commit details
    Browse the repository at this point in the history

  5. add geocode tale maker to makefile

    @eads
    eads committed Jul 11, 2018
    Configuration menu
    Copy the full SHA
    4620e37 View commit details
    Browse the repository at this point in the history

  6. error trapping, insert into unique db, and more fields in geocode ins… 

    …ertion
    @eads
    eads committed Jul 11, 2018
    Configuration menu
    Copy the full SHA
    62950cb View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2018

  1. use ranges, not offsets and limits, in geocoder. 

    also use update, TODO capture additional steps
    @eads
    eads committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    3e331e1 View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2018

  1. add some views

    @eads
    eads committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    b4b4d5a View commit details
    Browse the repository at this point in the history

  2. massive makefile overhaul 

    more declarative naming, views + community area stats, index and schema creation.
    @eads
    eads committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    bec3bc2 View commit details
    Browse the repository at this point in the history

  3. match addresses on ID when inserting, better guardrails on grabbing e… 

    …lements of geocoded data

ID matching is just more efficient since IDs are indexed and short. Some geocode's geojson objects don't have city, etc, so handled that was well.
    @eads
    eads committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    6adaf65 View commit details
    Browse the repository at this point in the history

  4. keep the dumps directory

    @eads
    eads committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    c4c1558 View commit details
    Browse the repository at this point in the history

  5. table definition for community area stats from CMAP

    @eads
    eads committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    4ac288f View commit details
    Browse the repository at this point in the history

  6. add amusingly embarassing note to readme

    @eads
    eads committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    5447790 View commit details
    Browse the repository at this point in the history

  7. Merge pull request #17 from propublica/0007-eads-geocoder 

    Geocoded data
    @eads
    eads authored Jul 13, 2018
    Configuration menu
    Copy the full SHA
    3f21496 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2018

  1. corrects = and :, cuts drop_db dependency on create_db, corrects refe… 

    …rence to URL when DB_NAME needed, closes #22
    kat-alo committed Jul 16, 2018
    Configuration menu
    Copy the full SHA
    018cbc4 View commit details
    Browse the repository at this point in the history

  2. adds csvkit requirement to README closes #23

    kat-alo committed Jul 16, 2018
    Configuration menu
    Copy the full SHA
    94d7e21 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2018

  1. replicate katlyn's makefile fixes

    @eads
    eads committed Jul 18, 2018
    Configuration menu
    Copy the full SHA
    8bd83a2 View commit details
    Browse the repository at this point in the history

  2. correct dependency in load_community_areas make command

    @eads
    eads committed Jul 18, 2018
    Configuration menu
    Copy the full SHA
    aff7213 View commit details
    Browse the repository at this point in the history

  3. fix community area matching, closes #25 

    o'hare and 'the loop' were causing problems, this rewrites them to correctly match across datasets
    @eads
    eads committed Jul 18, 2018
    Configuration menu
    Copy the full SHA
    f482c25 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #27 from propublica/0025-community-area-matching 

    fix community area matching
    @eads
    eads authored Jul 18, 2018
    Configuration menu
    Copy the full SHA
    9f681fc View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2018

  1. Merge branch 'master' of github.com:propublica/il-ticket-loader

    kat-alo committed Jul 19, 2018
    Configuration menu
    Copy the full SHA
    469fa3b View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2018

  1. load dump file

    @eads
    eads committed Jul 20, 2018
    Configuration menu
    Copy the full SHA
    55fb3d9 View commit details
    Browse the repository at this point in the history

  2. Merge branch '0025-community-area-matching'

    @eads
    eads committed Jul 20, 2018
    Configuration menu
    Copy the full SHA
    c7d0a8d View commit details
    Browse the repository at this point in the history

  3. ignore permissions from dump file, closes #30

    @eads
    eads committed Jul 20, 2018
    Configuration menu
    Copy the full SHA
    fabde92 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #32 from propublica/0030-pgrestore-perms 

    ignore permissions from dump file, closes #30
    @eads
    eads authored Jul 20, 2018
    Configuration menu
    Copy the full SHA
    f8f23bf View commit details
    Browse the repository at this point in the history

  5. add db string to environment

    @eads
    eads committed Jul 20, 2018
    Configuration menu
    Copy the full SHA
    9f4498f View commit details
    Browse the repository at this point in the history

  6. Merge pull request #33 from propublica/0031-dbstring 

    add db string to environment
    @eads
    eads authored Jul 20, 2018
    Configuration menu
    Copy the full SHA
    1878e72 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2018

  1. refactor make top level command signatures 

    consolidate 'bootstrap' and 'geo' into make targets
    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    a9a5c98 View commit details
    Browse the repository at this point in the history

  2. remove csvsql schema generation from makefile and turn into a note in… 

    … the readme3
    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    08d9f69 View commit details
    Browse the repository at this point in the history

  3. clean up restore permissions, change filename for now

    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    e8aa465 View commit details
    Browse the repository at this point in the history

  4. remove irrelevant notes file

    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    af0ff11 View commit details
    Browse the repository at this point in the history

  5. flip order of explanation about config

    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    fe65910 View commit details
    Browse the repository at this point in the history

  6. reflect latest make targets

    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    7e9aac2 View commit details
    Browse the repository at this point in the history

  7. handle 00s, closes #38

    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    1ec3af3 View commit details
    Browse the repository at this point in the history

  8. fix db url reference

    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    dc2e482 View commit details
    Browse the repository at this point in the history

  9. fix bad bug with wrong types for geocoder ranges

    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    07fd94d View commit details
    Browse the repository at this point in the history

  10. add quotes to community_area_stats table

    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    5bd3ece View commit details
    Browse the repository at this point in the history

  11. add geom column that's been giving us the flux

    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    328049a View commit details
    Browse the repository at this point in the history

  12. include just city sticker accuracy in geocode accuracy view

    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    46683fa View commit details
    Browse the repository at this point in the history

  13. add analysis view that joins every geocode with its community area

    @eads
    eads committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    8a91b61 View commit details
    Browse the repository at this point in the history

  14. Merge branch 'master' of github.com:propublica/il-ticket-loader

    kat-alo committed Jul 23, 2018
    Configuration menu
    Copy the full SHA
    76ff39c View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2018

  1. allows psql view to be updated when view is updated

    kat-alo committed Jul 25, 2018
    Configuration menu
    Copy the full SHA
    b75b2a1 View commit details
    Browse the repository at this point in the history

  2. adds median income to community area stats csv

    kat-alo committed Jul 25, 2018
    Configuration menu
    Copy the full SHA
    b41e4a6 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2018

  1. adds all years to data and salts n hashes the license plate numbers, c… 

    …loses #42, #43
    kat-alo committed Aug 21, 2018
    Configuration menu
    Copy the full SHA
    0bae864 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2018

  1. corrects naming of salt.txt file for the intermediate declaration I r… 

    …ealized it was wrong on the bus my b
    kat-alo committed Aug 22, 2018
    Configuration menu
    Copy the full SHA
    50cb16d View commit details
    Browse the repository at this point in the history

  2. Merge pull request #46 from propublica/0042-hash-plates 

    adds all years to data and salts n hashes the license plate numbers, …
    Kat-Alo authored Aug 22, 2018
    Configuration menu
    Copy the full SHA
    934ae71 View commit details
    Browse the repository at this point in the history

  3. exports parking table as one big csv and zips file, closes #50, closes … 

    …#49
    kat-alo committed Aug 22, 2018
    Configuration menu
    Copy the full SHA
    dedcd07 View commit details
    Browse the repository at this point in the history

  4. zip_n_ship makefile command now also exports zip to s3

    kat-alo committed Aug 22, 2018
    Configuration menu
    Copy the full SHA
    ceeafe6 View commit details
    Browse the repository at this point in the history

  5. forget to add the salt creator my b

    kat-alo committed Aug 22, 2018
    Configuration menu
    Copy the full SHA
    a5ecadc View commit details
    Browse the repository at this point in the history

  6. adds unit_key.csv to zip file to upload to s3

    kat-alo committed Aug 22, 2018
    Configuration menu
    Copy the full SHA
    55d484d View commit details
    Browse the repository at this point in the history

  7. adding unit_key.csv to data/ in case Nicolas Cage needs it

    kat-alo committed Aug 22, 2018
    Configuration menu
    Copy the full SHA
    2f64ebd View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2018

  1. Merge pull request #55 from propublica/0048-zip-n-ship-data 

    0048 zip n ship data, closes #48
    @eads
    eads authored Aug 23, 2018
    Configuration menu
    Copy the full SHA
    f16b11e View commit details
    Browse the repository at this point in the history

  2. Merge pull request #54 from propublica/0042-hash-plates 

    forget to add the salt creator my b
    @eads
    eads authored Aug 23, 2018
    Configuration menu
    Copy the full SHA
    e98910d View commit details
    Browse the repository at this point in the history

  3. adds data dict to zip file for s3

    kat-alo committed Aug 23, 2018
    Configuration menu
    Copy the full SHA
    36e35e2 View commit details
    Browse the repository at this point in the history

  4. adding requirements text file

    kat-alo committed Aug 23, 2018
    Configuration menu
    Copy the full SHA
    19d0577 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' of github.com:propublica/il-ticket-loader

    kat-alo committed Aug 23, 2018
    Configuration menu
    Copy the full SHA
    5d244de View commit details
    Browse the repository at this point in the history

  6. updating gitignore, closes #58

    kat-alo committed Aug 23, 2018
    Configuration menu
    Copy the full SHA
    aa1b7b1 View commit details
    Browse the repository at this point in the history

  7. updating readme to include data dictionary and python requirements, c… 

    …loses #52
    kat-alo committed Aug 23, 2018
    Configuration menu
    Copy the full SHA
    5c57664 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2018

  1. account for reason for dismissal from older data

    @eads
    eads committed Oct 12, 2018
    Configuration menu
    Copy the full SHA
    b3d54ef View commit details
    Browse the repository at this point in the history

  2. makefile cleanup

    @eads
    eads committed Oct 12, 2018
    Configuration menu
    Copy the full SHA
    aed0720 View commit details
    Browse the repository at this point in the history

  3. add dismissal reason to schema

    @eads
    eads committed Oct 12, 2018
    Configuration menu
    Copy the full SHA
    90d88b8 View commit details
    Browse the repository at this point in the history

  4. block summary 'views' (really derived tables until hasura supports ma… 

    …terialized views)
    @eads
    eads committed Oct 12, 2018
    Configuration menu
    Copy the full SHA
    2c071b7 View commit details
    Browse the repository at this point in the history

  5. fix view creation order

    @eads
    eads committed Oct 12, 2018
    Configuration menu
    Copy the full SHA
    25656dc View commit details
    Browse the repository at this point in the history

  6. update readme with additional reqs, view order

    @eads
    eads committed Oct 12, 2018
    Configuration menu
    Copy the full SHA
    e062fb3 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2018

  1. first tests. Found one that's not passing, too

    @eads
    eads committed Oct 18, 2018
    Configuration menu
    Copy the full SHA
    b3186b3 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2018

  1. trying out circleci

    @eads
    eads committed Oct 19, 2018
    Configuration menu
    Copy the full SHA
    4a7440d View commit details
    Browse the repository at this point in the history

  2. rename clean_csv test file for consistency

    @eads
    eads committed Oct 19, 2018
    Configuration menu
    Copy the full SHA
    709abcf View commit details
    Browse the repository at this point in the history

  3. add test for clean_row function

    @eads
    eads committed Oct 19, 2018
    Configuration menu
    Copy the full SHA
    0979214 View commit details
    Browse the repository at this point in the history

  4. use a config.yml that was autogenerated during project setup

    @eads
    eads committed Oct 19, 2018
    Configuration menu
    Copy the full SHA
    638bae7 View commit details
    Browse the repository at this point in the history

  5. match zero or more leading zeros from address strings

    @eads
    eads committed Oct 19, 2018
    Configuration menu
    Copy the full SHA
    0882b44 View commit details
    Browse the repository at this point in the history

  6. fix requirements for circle ci

    @eads
    eads committed Oct 19, 2018
    Configuration menu
    Copy the full SHA
    08704d0 View commit details
    Browse the repository at this point in the history

  7. Merge pull request #71 from propublica/0070-leading-zeroes 

    fix leading zeros, closes #71 

also set up circle ci
    @eads
    eads authored Oct 19, 2018
    Configuration menu
    Copy the full SHA
    52cbe31 View commit details
    Browse the repository at this point in the history

  8. Merge pull request #59 from propublica/0048-zip-n-ship-data 

    adds data dict to zip file for s3
    @eads
    eads authored Oct 19, 2018
    Configuration menu
    Copy the full SHA
    7ae0e2a View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2018

  1. derive geocodes from original table

    @eads
    eads committed Oct 22, 2018
    Configuration menu
    Copy the full SHA
    71e9744 View commit details
    Browse the repository at this point in the history

  2. try to speed up intermediate table query

    @eads
    eads committed Oct 22, 2018
    Configuration menu
    Copy the full SHA
    eb9d834 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2018

  1. more makefile optimization

    @eads
    eads committed Oct 23, 2018
    Configuration menu
    Copy the full SHA
    d86dacd View commit details
    Browse the repository at this point in the history

  2. make hash object once (cuts a few seconds off processing time)

    @eads
    eads committed Oct 23, 2018
    Configuration menu
    Copy the full SHA
    a3b141b View commit details
    Browse the repository at this point in the history

  3. use new year column in queries

    @eads
    eads committed Oct 23, 2018
    Configuration menu
    Copy the full SHA
    83ef995 View commit details
    Browse the repository at this point in the history

  4. commit index creating to sql files

    @eads
    eads committed Oct 23, 2018
    Configuration menu
    Copy the full SHA
    7cc4869 View commit details
    Browse the repository at this point in the history

  5. move test file back where it belongs

    @eads
    eads committed Oct 23, 2018
    Configuration menu
    Copy the full SHA
    230cc5f View commit details
    Browse the repository at this point in the history

  6. Merge pull request #76 from propublica/0074-dedupe-geocodes 

    dedupe geocodes better
    @eads
    eads authored Oct 23, 2018
    Configuration menu
    Copy the full SHA
    bf476ac View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2018

  1. use C collation 

    holy smokes
    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    7808e0b View commit details
    Browse the repository at this point in the history

  2. join through multiple addresses and lat / lngs

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    1211fcc View commit details
    Browse the repository at this point in the history

  3. the amount of branches with this fix is embarassing

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    2152bf8 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into 0078-extraction-tests

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    dd6c67b View commit details
    Browse the repository at this point in the history

  5. fix mistake in hashing

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    8e7611d View commit details
    Browse the repository at this point in the history

  6. clean up clean_csv file and factor date extractors into wee functions

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    9ccfef6 View commit details
    Browse the repository at this point in the history

  7. functional extraction tests

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    8dba7e9 View commit details
    Browse the repository at this point in the history

  8. Merge pull request #80 from propublica/0078-extraction-tests 

    date extraction tests, closes #78
    @eads
    eads authored Oct 24, 2018
    Configuration menu
    Copy the full SHA
    feb1f01 View commit details
    Browse the repository at this point in the history

  9. sane checks on existence of geocode tables and views

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    daaec3e View commit details
    Browse the repository at this point in the history

  10. introduce geocodes_normalized view

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    cfbbdef View commit details
    Browse the repository at this point in the history

  11. Merge pull request #84 from propublica/0077-partial-build-geo 

    partial build geodata
    @eads
    eads authored Oct 24, 2018
    Configuration menu
    Copy the full SHA
    91a9a72 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'master' into 0075-better-indexing

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    8490740 View commit details
    Browse the repository at this point in the history

  13. used normalized table consistently

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    171cf08 View commit details
    Browse the repository at this point in the history

  14. include geom in normalized geocode query

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    daddd0d View commit details
    Browse the repository at this point in the history

  15. Merge pull request #79 from propublica/0075-better-indexing 

    improve indexing, closes #75

with indexes + C collation, spatial queries that used to take minutes or more now take < 10s. databases are cool!
    @eads
    eads authored Oct 24, 2018
    Configuration menu
    Copy the full SHA
    577c675 View commit details
    Browse the repository at this point in the history

  16. write files to dupes dir, closes #88

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    f2c273f View commit details
    Browse the repository at this point in the history

  17. Merge pull request #91 from propublica/0088-fix-dupes 

    write files to dupes dir, closes #88
    @eads
    eads authored Oct 24, 2018
    Configuration menu
    Copy the full SHA
    1ba2fbd View commit details
    Browse the repository at this point in the history

  18. clean, correct, and add query example to readme 

    closes #89
    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    3bed0a4 View commit details
    Browse the repository at this point in the history

  19. Merge pull request #94 from propublica/0089-readme-cleanup 

    clean, correct, and add query example to readme
    @eads
    eads authored Oct 24, 2018
    Configuration menu
    Copy the full SHA
    553c94d View commit details
    Browse the repository at this point in the history

  20. fix bad data dict list in readme

    @eads
    eads committed Oct 24, 2018
    Configuration menu
    Copy the full SHA
    a378d52 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2018

  1. add violation lookup

    @eads
    eads committed Oct 25, 2018
    Configuration menu
    Copy the full SHA
    b73b131 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #96 from propublica/0092-violation-lookup 

    add violation lookup
    @eads
    eads authored Oct 25, 2018
    Configuration menu
    Copy the full SHA
    615be24 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2018

  1. wip

    @eads
    eads committed Oct 29, 2018
    Configuration menu
    Copy the full SHA
    e0a87e6 View commit details
    Browse the repository at this point in the history

  2. new core DB structure for ward analysis

    @eads
    eads committed Oct 29, 2018
    Configuration menu
    Copy the full SHA
    ba35118 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #97 from propublica/0081-ward-analysis-work-backward 

    better ward analysis
    @eads
    eads authored Oct 29, 2018
    Configuration menu
    Copy the full SHA
    c494a93 View commit details
    Browse the repository at this point in the history

  4. clean up, skip clustering for now

    @eads
    eads committed Oct 29, 2018
    Configuration menu
    Copy the full SHA
    1facd58 View commit details
    Browse the repository at this point in the history

  5. Merge pull request #100 from propublica/0092-violation-lookup 

    add violation lookup
    @eads
    eads authored Oct 29, 2018
    Configuration menu
    Copy the full SHA
    7d6b7e9 View commit details
    Browse the repository at this point in the history

  6. load metadata

    @eads
    eads committed Oct 29, 2018
    Configuration menu
    Copy the full SHA
    430e29d View commit details
    Browse the repository at this point in the history

  7. Merge pull request #101 from propublica/0099-ward-meta 

    load metadata for wards
    @eads
    eads authored Oct 29, 2018
    Configuration menu
    Copy the full SHA
    d7ef46b View commit details
    Browse the repository at this point in the history

Commits on Nov 5, 2018

  1. calculate penalties; refactor data structure

    @eads
    eads committed Nov 5, 2018
    Configuration menu
    Copy the full SHA
    ca552da View commit details
    Browse the repository at this point in the history

  2. Merge pull request #108 from propublica/0105-calculate-penalties 

    calculate penalties; refactor data structure
    @eads
    eads authored Nov 5, 2018
    Configuration menu
    Copy the full SHA
    4244acc View commit details
    Browse the repository at this point in the history

  3. ignore mapbox

    @eads
    eads committed Nov 5, 2018
    Configuration menu
    Copy the full SHA
    b6fad26 View commit details
    Browse the repository at this point in the history

  4. restore parking

    @eads
    eads committed Nov 5, 2018
    Configuration menu
    Copy the full SHA
    8e2ae66 View commit details
    Browse the repository at this point in the history

  5. Merge pull request #111 from propublica/0110-parking 

    restore parking
    @eads
    eads authored Nov 5, 2018
    Configuration menu
    Copy the full SHA
    a3b3e22 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2018

  1. upload data to mapbox

    @eads
    eads committed Nov 7, 2018
    Configuration menu
    Copy the full SHA
    892e71c View commit details
    Browse the repository at this point in the history

  2. refactor penalty calculator a bit

    @eads
    eads committed Nov 7, 2018
    Configuration menu
    Copy the full SHA
    683e2bb View commit details
    Browse the repository at this point in the history

  3. move hasura over here

    @eads
    eads committed Nov 7, 2018
    Configuration menu
    Copy the full SHA
    e6ad632 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #112 from propublica/0098-hasura 

    move hasura over here
    @eads
    eads authored Nov 7, 2018
    Configuration menu
    Copy the full SHA
    23f714e View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2018

  1. make geo layers for app

    @eads
    eads committed Nov 28, 2018
    Configuration menu
    Copy the full SHA
    4b37297 View commit details
    Browse the repository at this point in the history

  2. calculate % successfully contested

    @eads
    eads committed Nov 28, 2018
    Configuration menu
    Copy the full SHA
    4426bf3 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2018

  1. add make command to populate geocode table

    @eads
    eads committed Nov 29, 2018
    Configuration menu
    Copy the full SHA
    07aa9e6 View commit details
    Browse the repository at this point in the history

  2. clean up makefile

    @eads
    eads committed Nov 29, 2018
    Configuration menu
    Copy the full SHA
    61bb0a4 View commit details
    Browse the repository at this point in the history

  3. update db schema and indexes for geocodio

    @eads
    eads committed Nov 29, 2018
    Configuration menu
    Copy the full SHA
    bc8e146 View commit details
    Browse the repository at this point in the history

  4. add scrapy geocoder

    @eads
    eads committed Nov 29, 2018
    Configuration menu
    Copy the full SHA
    f6ce55e View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2018

  1. functional new geocoder

    @eads
    eads committed Nov 30, 2018
    Configuration menu
    Copy the full SHA
    6f17159 View commit details
    Browse the repository at this point in the history

  2. remove old geocoder

    @eads
    eads committed Nov 30, 2018
    Configuration menu
    Copy the full SHA
    3118910 View commit details
    Browse the repository at this point in the history

  3. start loading census data

    @eads
    eads committed Nov 30, 2018
    Configuration menu
    Copy the full SHA
    0e24589 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2018

  1. wip, functional new geocoding

    @eads
    eads committed Dec 6, 2018
    Configuration menu
    Copy the full SHA
    6b4b389 View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2018

  1. tweaks for app, tests

    @eads
    eads committed Dec 10, 2018
    Configuration menu
    Copy the full SHA
    119885b View commit details
    Browse the repository at this point in the history

  2. test % of debt captured in wards vs city wide figures

    @eads
    eads committed Dec 10, 2018
    Configuration menu
    Copy the full SHA
    f77b8eb View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2018

  1. use lte and gte for totals query

    @eads
    eads committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    a0846cd View commit details
    Browse the repository at this point in the history

  2. better block query

    @eads
    eads committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    fa1e498 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2018

  1. preliminary bulletproofing -- to be pipelined once we lock down details

    @j2kao
    j2kao committed Dec 12, 2018
    Configuration menu
    Copy the full SHA
    f15946c View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2018

  1. lots of small updates and fixes

    @eads
    eads committed Dec 13, 2018
    Configuration menu
    Copy the full SHA
    3d24b4f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into 0127-bulletproof

    @eads
    eads committed Dec 13, 2018
    Configuration menu
    Copy the full SHA
    cabc124 View commit details
    Browse the repository at this point in the history

Commits on Dec 17, 2018

  1. tons of small data fixes

    @eads
    eads committed Dec 17, 2018
    Configuration menu
    Copy the full SHA
    dba762c View commit details
    Browse the repository at this point in the history

  2. bump to latest metadata

    @eads
    eads committed Dec 17, 2018
    Configuration menu
    Copy the full SHA
    f05a016 View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2018

  1. check top 5 violation categories for 2013-2017

    @j2kao
    j2kao committed Dec 18, 2018
    Configuration menu
    Copy the full SHA
    49b7a46 View commit details
    Browse the repository at this point in the history

  2. add top 5 and top level summaries

    @j2kao
    j2kao committed Dec 18, 2018
    Configuration menu
    Copy the full SHA
    8cb830c View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2018

  1. many data fixes for production

    @eads
    eads committed Dec 20, 2018
    Configuration menu
    Copy the full SHA
    d589fd9 View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2019

  1. fix camera loader to accomodate older data

    @eads
    eads committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    2749c64 View commit details
    Browse the repository at this point in the history

  2. do the pipfile thing

    @eads
    eads committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    69d3023 View commit details
    Browse the repository at this point in the history

  3. camera data now runs through python processor and needs new columns

    @eads
    eads committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    7aa1029 View commit details
    Browse the repository at this point in the history

  4. remove duplicate issue date in table definition

    @eads
    eads committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    b80f589 View commit details
    Browse the repository at this point in the history

  5. add some exports for block club

    @eads
    eads committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    5694556 View commit details
    Browse the repository at this point in the history

  6. ignore bulletproofing stuff

    @eads
    eads committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    fc58842 View commit details
    Browse the repository at this point in the history

  7. make datastore directory

    @eads
    eads committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    27c31dc View commit details
    Browse the repository at this point in the history

  8. use exports framework to zip-n-ship

    @eads
    eads committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    7e69e02 View commit details
    Browse the repository at this point in the history

  9. add makefile download alias

    @eads
    eads committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    7c7087c View commit details
    Browse the repository at this point in the history

  10. update README for release

    @eads
    eads committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    2452f30 View commit details
    Browse the repository at this point in the history

  11. add exports and turn off s3 uploading for now

    @eads
    eads committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    7428b11 View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2019

  1. add bulletproofing stuff to messy gitignore

    @eads
    eads committed Mar 4, 2019
    Configuration menu
    Copy the full SHA
    85ac89a View commit details
    Browse the repository at this point in the history

  2. merge in way that should allow for merging back to master

    @eads
    eads committed Mar 4, 2019
    Configuration menu
    Copy the full SHA
    7c21419 View commit details
    Browse the repository at this point in the history