The Battle of Neighborhoods (Week 1).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A data scientist currently residing in Downtown Singapore. \n",
    "He currently live within walking distance to Downtown \"Telok Ayer MRT metro station\" therefore He has access to good public transportation to work. Likewise, He enjoy many ammenities in the neighborhood , such as international cousine restaurants, cafes, food shops and entertainment.\n",
    "He has been offered a great opportunity to work in Manhattan, NY. Although, He is very excited about it, He is a bit stress toward the process to secure a comparable place to live in Manhattan. Therefore, He decided to apply the learned skills during the Coursera course to explore ways to make sure his decision is factual and rewarding.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The challenge to resolve is being able to find a rental apartment unit in Manhattan NY that offers similar characteristics and benefits to my current situation. Therefore, in order to set a basis for comparison, we want to find a renta unit subject to the following conditions: \n",
    "- Apartment with min 2 bedrooms with monthly rent not to exceed US$7000/month\n",
    "- Unit located within walking distance (<=1.0 mile, 1.6 km) from a subway metro station in Manhattan\n",
    "- Area with ammenities and venues similar to the ones described for the location \n",
    "\n",
    "### Interested Audience\n",
    "I believe this is a relevant project for a person or entity considering moving to a major city in Europe, US or Asia, since the approach and methodologies used here are applicable in all cases. The use of FourSquare data and mapping techniques combined with data analysis will help resolve the key questions arisen. Lastly, this project is a good practical case toward the development of Data Science skills."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Section"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to make a good choice of a similar apartment in Manhattan NY, the following data is required:\n",
    "List/Information on neighborhoods form Manhattan with their Geodata ( latitud and longitud.\n",
    "List/Information about the subway metro stations in Manhattan with geodata.\n",
    "Listed apartments for rent in Manhattan area with descriptions ( how many beds, price, location, address)\n",
    "Venues and ammenities in the Manhattan neighborhoods (e.g. top 10)\n",
    "\n",
    "#### sources \n",
    "The list of Manhattan neighborhoods is worked out during LAb exercise during the course. A csv file was created which will be read in order to create a dataframe and its mapping. The csv file 'mh_neigh_data.csv'  has the following below data structure. The file will be directly read to the Jupiter Notebook for convenience and space savings. The clustering of neighborhoods and mapping will be shown however. An algorythm was used to determine the geodata from Nominatim . The actual algorythm coding may be shown in 'markdown' mode becasues it takes time to run. \n",
    "\n",
    "mh_neigh_data.tail():\n",
    "\n",
    "              Borough    Neighborhood    Latitude    Longitude\n",
    "\n",
    "          35    Manhattan    Turtle Bay    40.752042    -73.967708\n",
    "\n",
    "          36    Manhattan    Tudor City    40.746917    -73.971219\n",
    "\n",
    "          37    Manhattan    Stuyvesant Town    40.731000    -73.974052\n",
    "\n",
    "          38    Manhattan    Flatiron    40.739673    -73.990947\n",
    "\n",
    "          39    Manhattan    Hudson Yards    40.756658    -74.000111\n",
    "  \n",
    "A list of Manhattan subway metro stops was complied in Numbers (Apple excel) and it was complemeted with wikipedia data ( https://en.wikipedia.org/wiki/List_of_New_York_City_Subway_stations_in_Manhattan) and information from NY Transit authority and Google maps (https://www.google.com/maps/search/manhattan+subway+metro+stations/@40.7837297,-74.1033043,11z/data=!3m1!4b1) for a final consolidated list of subway stops names and their address. The geolocation was obtained via an algorythm using Nominatim. Details will be shown in the execution of methodolody in section 3.0. The subway csv file is ''MH_subway.csv'\" and the data structure is: mhsub.tail():\n",
    "               sub_station                    sub_address                         lat        long\n",
    "\n",
    "      17    190 Street Subway Station    Bennett Ave, New York, NY 10040, USA    40.858113    -73.932983\n",
    "\n",
    "      18    59 St-Lexington Av Station    E 60th St, New York, NY 10065, USA        40.762259    -73.966271\n",
    "\n",
    "      19    57 Street Station    New York, NY 10019, United States                40.764250    -73.954525\n",
    "\n",
    "      20    14 Street / 8 Av    New York, NY 10014, United States                40.730862    -73.987156\n",
    "\n",
    "      21    MTA New York City    525 11th Ave, New York, NY 10018, USA            40.759809    -73.999282\n",
    "A list of places for rent was collected by web-browsing real estate companies in Manhattan : http://www.rentmanhattan.com/index.cfm?page=search&state=results https://www.nestpick.com/search?city=new-york&page=1&order=relevance&district=manhattan&gclid=CjwKCAiAjNjgBRAgEiwAGLlf2hkP3A-cPxjZYkURqQEswQK2jKQEpv_MvKcrIhRWRzNkc_r-fGi0lxoCA7cQAvD_BwE&type=apartment&display=list https://www.realtor.com/apartments/Manhattan_NY\n",
    "A csv file was compiled with the rental place that indicated: areas of Manhattan, address, number of beds, area and monthly rental price. The csv file \"nnnn.csv\" had the following below structure. An algorythm was used to create all the geodata using Nominatim, as shown in section 3.0. The actual algorythm coding may be shown in 'markdown' mode becasues it takes time to run. With the use of geolocator = Nominatim() , it was possible to determine the latitude and longiude for the subway metro locations as well as for the geodata for each rental place listed. The loop algorythms used are shown in the execution of data in section 3.0\n",
    "\"Great_circle\" function from geolocator was used to calculate distances between two points , as in the case to calculate average rent price for units around each subway station and at 1.6 km radius.\n",
    "Foursquare is used to find the avenues at Manhattan neighborhoods in general and a cluster is created to later be able to search for the venues depending of the location shown.\n",
    "\n",
    "#### How the data will be used to solve the problem\n",
    "The data will be used as follows:\n",
    "Use Foursquare and geopy data to map top 10 venues for all Manhattan neighborhoods and clustered in groups ( as per Course LAB)\n",
    "Use foursquare and geopy data to map the location of subway metro stations , separately and on top of the above clustered map in order to be able to identify the venues and ammenities near each metro station, or explore each subway location separately\n",
    "Use Foursquare and geopy data to map the location of rental places, in some form, linked to the subway locations.\n",
    "create a map that depicts, for instance, the average rental price per square ft, around a radious of 1.0 mile (1.6 km) around each subway station - or a similar metrics. I will be able to quickly point to the popups to know the relative price per subway area.\n",
    "Addresses from rental locations will be converted to geodata( lat, long) using Geopy-distance and Nominatim.\n",
    "Data will be searched in open data sources if available, from real estate sites if open to reading, libraries or other government agencies such as Metro New York MTA, etc.\n",
    "\n",
    "#### Mapping of Data\n",
    "The following maps were created to facilitate the analysis and the choice of the palace to live.\n",
    "Manhattan map of Neighborhoods\n",
    "manhattan subway metro locations\n",
    "Manhattan map of places for rent\n",
    "Manhattan map of clustered venues and neighborhoods\n",
    "Combined maps of Manhattan rent places with subway locations\n",
    "Combined maps of Manhattan rent places with subway locations and venues clusters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}