Table of Contents

Foreword
Introduction
1 Scope
2 Normative references
3 Terms, definitions and abbreviated terms
4 File and record model
5 Named fields
6 WARC record types
7 Record segmentation
8 WARC file name, size and compression
Annex A (informative) - Use cases for writing WARC
        records
Annex B (informative) - Examples of WARC records
Annex C (informative) - WARC file size and name
        recommendations
Annex D (informative) - Compression recommendations
Bibliography

Abstract

Defines the WARC file format: - to store both the payload content and control information from mainstream Internet application layer protocols, such as the HTTP, DNS, and FTP; - to store arbitrary metadata linked to other stored data (e.g. subject classifier, discovered language, encoding); - to support data compression and maintain data record integrity; - to store all control information from the harvesting protocol (e.g. request headers), not just response information; - to store the results of data transformations linked to other stored data; - to store a duplicate detection event linked to other stored data (to reduce storage in the presence of identical or substantially similar resources); - to be extended without disruption to existing functionality; - to support handling of overly long records by truncation or segmentation, where desired.

General Product Information

Published
Document Type	Standard
Status	Current
Publisher	International Organization for Standardization
Pages
ISBN
Committee	TC 46
Supersedes	ISO/DIS 28500 : 60.00 (2017)