2024-11-12 05:17:45 +01:00
|
|
|
# Web scraper 🔍
|
|
|
|
|
|
|
|
|
|
## Description
|
|
|
|
|
|
2024-11-15 17:13:29 +01:00
|
|
|
This project is a web scraper designed to extract data from websites.
|
2024-11-12 05:17:45 +01:00
|
|
|
|
2025-01-28 16:26:29 +01:00
|
|
|
## How to use
|
|
|
|
|
|
|
|
|
|
1. Clone the repository
|
|
|
|
|
1. `cd webscraper`
|
|
|
|
|
1. `cd app`
|
|
|
|
|
1. `pip3 install -r requirements.txt`
|
|
|
|
|
1. `python3 scripts/monet.py` for the monet scraper
|
|
|
|
|
1. `python3 scripts/torres.py` for the torres scraper
|
2025-01-28 16:29:43 +01:00
|
|
|
|
|
|
|
|
## How to use with Docker
|
|
|
|
|
|
|
|
|
|
1. Clone the repository
|
|
|
|
|
1. `cd webscraper/app`
|
|
|
|
|
1. `docker compose up -d`
|
|
|
|
|
1. `docker exec -it docker exec -it scraper xvfb-run --auto-servernum --server-num=1 --server-args='-screen 0, 1920x1080x24' python3 scripts/monet.py` for the monet scraper
|
|
|
|
|
1. `docker exec -it docker exec -it scraper xvfb-run --auto-servernum --server-num=1 --server-args='-screen 0, 1920x1080x24' python3 scripts/torres.py` for the torres scraper
|
|
|
|
|
1. `docker compose down`
|
|
|
|
|
|
2025-01-28 17:16:29 +01:00
|
|
|
## Video
|
|
|
|
|
|
2025-01-28 17:17:10 +01:00
|
|
|
[Watch the video](https://drive.google.com/drive/folders/1BrLMTmlHGz7__cs6-g-UorUgbfC-sSXG?usp=drive_link)
|
