Expand description
Warcat: WARC Archiving Tool
This crate provides both a library API and a binary CLI application. The library can be used to read and write WARC files and as well perform functions provided by the binary.
In general cases, users working with WARC files do not need to program directly with the library. The CLI application (the tool portion) is designed to be part of a Unix-style pipeline.
This documentation is for the library portion. For details on the CLI, see the user guide.
The library is designed first in mind for the binary, so some parts of the API will be unstable or not relevant.
The main entrypoints to this library is warc::Decoder/warc::PushDecoder and warc::Encoder.
Modulesยง
- compress
- Reader and writer abstractions for compressed streams.
- dataseq
- Streams of serialized values.
- digest
- WARC related hashing functions.
- error
- Error representations
- extract
- Content extraction from WARC files
- fields
- HTTP-style name-value fields
- header
- WARC headers
- http
- Things dealing with the HTTP protocol
- io
- IO utilities
- parse
- Parsing utilities.
- verify
- WARC file verification.
- warc
- WARC file format