Backend Challenge

A web scraping and data storage service using FastAPI, SQLModel, and PostgreSQL to extract product data from various supermarket websites and store it in a database.

Prerequisites

• Docker
• Docker Compose

Steps

1. Clone the Repository

   Clone the repository to your local machine:

   git clone https://github.com/LeanSerra/BackendChallenge.git
   cd BackendChallenge
   

2. Create a .env file

   Create a .env file in the root directory and provide a PostgreSQL connection string

   DATABASE_URL=postgresql://postgres:password@postgres:5432/
   

3. Make entrypoint.sh executable

   chmod +x entrypoint.sh

4. Start Docker containers

   docker compose up

5. (Optional) Preload the db with common keywords

   chmod +x populate_db.sh
   ./populate_db.sh

6. (Optional) Run tests

   pytest

Usage

• After deployment, the API can be accessed locally at http://localhost:8000.
• Swagger documentation is available at http://localhost:8000/docs