Crawler

A naive web crawler written in C. To compile and use it, make sure libglib2.0-dev is installed.

Notice

No caching, file whose size greater than 500k is truncated.

Name		Name	Last commit message	Last commit date
Latest commit History 20 Commits
Makefile		Makefile
README.md		README.md
crawler_core.c		crawler_core.c
crawler_core.h		crawler_core.h
file_proc.c		file_proc.c
file_proc.h		file_proc.h
http_parser.c		http_parser.c
http_parser.h		http_parser.h
http_requester.c		http_requester.c
http_requester.h		http_requester.h
main.c		main.c
queue.c		queue.c
queue.h		queue.h