mass-scraper/environment.yml

name: polish-masses
channels:
  - defaults
  - conda-forge
dependencies:
  - python
  - scrapy
  - gunicorn
  - flask
  - pip:
    - lxml
    - tldextract
    - requests
    - beautifulsoup4
    - python-google-places
    - jsonlines
    - ipdb
    - colorama
    - html2text
    - binaryornot
Prototype rule based masses extractor. Added spider. Started working on testsets. 2018-01-20 21:55:26 +01:00			`name: polish-masses`
			`channels:`
Add domain-blacklist.txt, domain filter, modify crawler. Add binary or not checker. 2018-04-09 23:52:11 +02:00			`- defaults`
Prototype rule based masses extractor. Added spider. Started working on testsets. 2018-01-20 21:55:26 +01:00			`- conda-forge`
			`dependencies:`
			`- python`
			`- scrapy`
Add basic wsgi app. Rename extractors, change directories. Add gunicorn and flask to environment.yml Update .gitignore 2018-04-27 22:44:15 +02:00			`- gunicorn`
			`- flask`
Prototype rule based masses extractor. Added spider. Started working on testsets. 2018-01-20 21:55:26 +01:00			`- pip:`
			`- lxml`
			`- tldextract`
			`- requests`
			`- beautifulsoup4`
			`- python-google-places`
			`- jsonlines`
			`- ipdb`
			`- colorama`
Switch to pure html download. Enhanced urls filtering. Update Makefile. 2018-03-11 18:02:31 +01:00			`- html2text`
Add domain-blacklist.txt, domain filter, modify crawler. Add binary or not checker. 2018-04-09 23:52:11 +02:00			`- binaryornot`