webscraper/README.md

Web scraper 🔍

Description

This project is a web scraper designed to extract data from websites.

Features

☑️ Extracts data from web pages

Usage

With Docker

1. Clone the repository:

git clone https://git.wmi.amu.edu.pl/s500042/webscraper

2. Navigate to the project directory:

cd webscraper

3. Build the Docker image and run it using start.py script:

python scripts/start.py

On Mac, you'll have to use

python3 scripts/start.py

4. Check /app/dist/data.json file to see the extracted data.

Without Docker

1. Clone the repository:

git clone https://git.wmi.amu.edu.pl/s500042/webscraper

2. Install the required dependencies:

pip install -r app/requirements.txt

If you're on Arch Linux, you'll need to create a virtual environment. Here's is a Step by step guide that will help you create it.

3. Run run_with_no_docker.py script:

python scripts/run_with_no_docker.py

On Mac you'll, need to use:

python3 scripts/run_with_no_docker.py

4. Check /app/dist/data.json file to see the extracted data.

License

This project is licensed under the MIT License. See the LICENSE file for details.