TOC 
Network Working GroupL. Degioanni
Internet-DraftF. Risso
Expires: August 30, 2004Politecnico di Torino
 March 2004

PCAP Next Generation Dump File Format

PCAP-DumpFileFormat2

Status of this Memo

This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.

The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.

This Internet-Draft will expire on August 30, 2004.

Copyright Notice

Copyright (C) The Internet Society (2004). All Rights Reserved.

Abstract

This document describes a format to dump captured packets on a file. This format is extensible and it is currently proposed for implementation in the libpcap/WinPcap packet capture library.



Table of Contents

1.  Objectives
2.  General File Structure
    2.1  General Block Structure
    2.2  Block Types
    2.3  Block Hierarchy and Precedence
    2.4  Data format
3.  Block Definition
    3.1  Section Header Block (mandatory)
    3.2  Interface Description Block (mandatory)
    3.3  Packet Block (optional)
    3.4  Simple Packet Block (optional)
    3.5  Name Resolution Block (optional)
    3.6  Interface Statistics Block (optional)
4.  Options
5.  Experimental Blocks (deserved to a further investigation)
    5.1  Other Packet Blocks (experimental)
    5.2  Compression Block (experimental)
    5.3  Encryption Block (experimental)
    5.4  Fixed Length Block (experimental)
    5.5  Directory Block (experimental)
    5.6  Traffic Statistics and Monitoring Blocks (experimental)
    5.7  Event/Security Block (experimental)
6.  Conclusions
7.  Most important open issues
A.  Packet Block Flags Word
§  Intellectual Property and Copyright Statements




 TOC 

1. Objectives

The problem of exchanging packet traces becomes more and more critical every day; unfortunately, no standard solutions exist for this task right now. One of the most accepted packet interchange formats is the one defined by libpcap, which is rather old and does not fit for some of the nowadays applications particularly from the extensibility point of view.

This document proposes a new format for dumping packet traces. The following goals are being pursued:



 TOC 

2. General File Structure

2.1 General Block Structure

A capture file is organized in blocks, that are appended one to another to form the file. All the blocks share a common format, which is shown in Figure 1Basic block structure..



    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Block Type                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Block Total Length                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   /                          Block Body                           /
   /          /* variable length, aligned to 32 bits */            /
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Block Total Length                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 Basic block structure. 

The fields have the following meaning:

This structure, shared among all blocks, makes easy to process a file and to skip unneeded or unknown blocks. Some blocks can contain other blocks inside (nested blocks). Some of the blocks are mandatory, i.e. a dump file is not valid if they are not present, other are optional.

The General Block Structure allows defining other blocks if needed. A parser that does non understand them can simply ignore their content.

2.2 Block Types

Blocks currently defined are the following:

  1. Section Header Block: it defines the most important characteristics of the capture file.
  2. Interface Description Block: it defines the most important characteristics of the interface(s) used for capturing traffic.
  3. Packet Block: it contains a single captured packet, or a portion of it.
  4. Simple Packet Block: it contains a single captured packet, or a portion of it, with only a minimal set of information about it.
  5. Name Resolution Block: it defines the mapping from numeric addresses present in the packet dump and the canonical name counterpart.
  6. Capture Statistics Block: it defines how to store some statistical data (e.g. packet dropped, etc) which can be useful to undestand the conditions in which the capture has been made.

The following blocks instead are considered interesting but the authors believe that they deserve more in-depth discussion before being defined:

  1. Additional Packet Blocks
  2. Compression Marker Block
  3. Encryption Marker Block
  4. Fixed Length Marker Block
  5. Directory Block
  6. Traffic Statistics and Monitoring Blocks
  7. Event/Security Blocks

Currently standardized Block Type codes are specified in Appendix 1. (TODO)

2.3 Block Hierarchy and Precedence

The file must begin with a Section Header Block. However, more than one Section Header Block can be present on the dump, each one covering the data following it till the next one (or the end of file). A Section includes the data delimited by two Section Header Blocks (or by a Section Header Block and the end of the file), including the first Section Header Block.

In case an application cannot read a Section because of different version number, it must skip everything until the next Section Header Block. Note that, in order to properly skip the blocks until the next section, all blocks must have the fields Type and Length at the beginning. This is a mandatory requirement that must be maintained in future versions of the block format.

Figure 2File structure example: the Section Header Block. shows two valid files: the first has a typical configuration, with a single Section Header that covers the whole file. The second one contains three headers, and is normally the result of file concatenation. An application that understands only version 1.0 of the file format skips the intermediate section and restart processing the packets after the third Section Header.



   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | SHB v1.0  |                      Data                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   Typical configuration with a single Section Header Block 


   |--   1st Section   --|--   2nd Section   --|--  3rd Section  --|
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | SHB v1.0  |  Data   | SHB V1.1  |  Data   | SHB V1.0  |  Data |  
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   Configuration with three different Section Header Blocks
 File structure example: the Section Header Block. 

NOTE: TO BE COMPLETED with some examples of other blocks

2.4 Data format

Data contained in each section will always be saved according to the characteristics (little endian / big endian) of the dumping machine. This refers to all fields that are saved as numbers and that span over two or more bytes.

The approach of having each section saved in the native format of the generating host is more efficient because it avoids translation of data when reading / writing on the host itself, which is the most common case when generating/processing capture dumps.

TODO Probably we have to specify something more here. Is what we're saying enough to avoid any kind of ambiguity?.



 TOC 

3. Block Definition

This section details the format of the body of the blocks currently defined.

3.1 Section Header Block (mandatory)

The Section Header Block is mandatory. It identifies the beginning of a section of the capture dump file. Its format is shown in Figure 3Section Header Block format..



    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Byte-Order Magic                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Major Version        |         Minor Version         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                          Section Length                       |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   /                                                               / 
   /                      Options (variable)                       / 
   /                                                               / 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 Section Header Block format. 

The meaning of the fields is:

Adding new block types or options would not necessarily require that either Major or Minor numbers be changed, as code that does not know about the block type or option could just skip it; only if skipping a block or option does not work should the minor version number be changed.

The block type of the Section Header Block is the integer corresponding to the 4-char string "\r\n\n\r". This particular value is used for 2 reasons:

  1. This number is used to detect if a file has been trasferred via FTP from a machine to another without the proper ASCII conversion. In this case, the value if this field will differ from the standard one ("\r\n\n\r") and the reader can detect a possibly corrupted file.
  2. This value is palindromic, so that the reader is able to recognize the Section Header Block regardless of the endianess of the section. The endianess is recognized by reading the Byte Order Magic, that is located 8 bytes after the Block Type.

Aside from the options defined in Section 4Options, the following options are valid within this block:

Name Code Length Description
Hardware 2 variable An UTF-8 string containing the description of the hardware used to create this section.
Operating System 3 variable An UTF-8 string containing the name of the operating system used to create this section.
User Application 4 variable An UTF-8 string containing the name of the application used to create this section.

The Section Header Block does not contain data but it rather identifies a list of blocks (interfaces, packets) that are logically correlated.

Special care should be taken in accessing the 64-bit field Section Length: since the alignment of all the blocks in the file is 32-bit, this field is not guaranteed to be aligned to a 64-bit boundary. This could be a problem on 64-bit workstations.

3.2 Interface Description Block (mandatory)

The Interface Description Block is mandatory. This block is needed to specify the characteristics of the network interface on which the capture has been made. In order to properly associate the captured data to the corresponding interface, the Interface Description Block must be defined before any other block that uses it; therefore, this block is usually placed immediately after the Section Header Block.

An Interface Description Block is valid only inside the section which it belongs to. The structure of a Interface Description Block is shown in Figure 4Interface Description Block format..



    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           LinkType            |           Reserved            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                            SnapLen                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   /                                                               / 
   /                      Options (variable)                       / 
   /                                                               / 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 
 Interface Description Block format. 

The meaning of the fields is:

Tools that write / read the capture file associates a progressive 16-bit number (starting from '0') to each Interface Definition Block. This number is unique within each Section and identifies uniquely the interface (inside current section); therefore, two Sections can have interfaces identified by the same identifiers. This unique identifier is referenced by other blocks (e.g. Packet Block) to point out the interface the block refers to (e.g. the interface that was used to capture the packet).

In addition to the options defined in Section 4Options, the following options are valid within this block:

Name Code Length Description
if_name 2 Variable A UTF-8 string containing the name of the device used to capture data.
if_description 3 Variable A UTF-8 string containing the description of the device used to capture data.
if_IPv4addr 4 8 Interface network address and netmask.
if_IPv6addr 5 17 Interface network address and prefix length (stored in the last byte).
if_MACaddr 6 6 Interface Hardware MAC address (48 bits).
if_EUIaddr 7 8 Interface Hardware EUI address (64 bits), if available.
if_speed 8 8 Interface speed (in bps).
if_tsaccur 9 1 Precision of timestamps. If the Most Significant Bit is equal to zero, the remaining bits indicates the accuracy as as a negative power of 10 (e.g. 6 means microsecond accuracy). If the Most Significant Bit is equal to zero, the remaining bits indicates the accuracy as as negative power of 2 (e.g. 10 means 1/1024 of second). If this option is not present, a precision of 10^-6 is assumed.
if_tzone 10 4 Time zone for GMT support (TODO: specify better).
if_filter 11 variable The filter (e.g. "capture only TCP traffic") used to capture traffic. The first byte of the Option Data keeps a code of the filter used (e.g. if this is a libpcap string, or BPF bytecode, and more). More details about this format will be presented in Appendix XXX (TODO).
if_os 12 variable A UTF-8 string containing the name of the operating system of the machine in which this interface is installed. This can be different from the same information that can be contained by the Section Header Block (Section 3.1Section Header Block (mandatory)) because the capture can have been done on a remote machine.
if_fcslen 13 1 An integer value that specified the length of the Frame Check Sequence (in bits) for this interface. For link layers whose FCS length can change during time, the Packet Block Flags Word can be used (see Appendix 1).

3.3 Packet Block (optional)

A Packet Block is the standard container for storing the packets coming from the network. The Packet Block is optional because packets can be stored either by means of this block or the Simple Packet Block, which can be used to speed up dump generation. The format of a packet block is shown in Figure 5Packet Block format..



    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Interface ID          |          Drops Count          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Timestamp (High)                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Timestamp (Low)                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         Captured Len                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Packet Len                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   /                                                               /
   /                          Packet Data                          /
   /          /* variable length, aligned to 32 bits */            /
   /                                                               /
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   /                                                               / 
   /                      Options (variable)                       / 
   /                                                               / 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 Packet Block format. 

The Packet Block has the following fields:

In addition to the options defined in Section 4Options, the following options are valid within this block:

Name Code Length Description
pack_flags 2 4 A flags word containing link-layer information. A complete specification of the allowed flags can be found in appendix A>.
pack_hash 4 variable This option contains a hash of the packet. The first bit specifies the hashing algorithm, while the following bytes contain the actual hash, whose size depends on the hashing algorithm, and hence from the value in the first bit. The hashing algorithm can be: 2s complement (algorithm byte = 0, size=XXX), XOR (algorithm byte = 1, size=XXX), CRC32 (algorithm byte = 2, size = 4), MD-5 (algorithm byte = 3, size=XXX), SHA-1 (algorithm byte = 4, size=XXX). The hash covers only the packet, not the header added by the capture driver: this gives the possibility to calculate it inside the network card. The hash allows easier comparison/merging of different capture files, and reliable data transfer between the data acquisition system and the capture library.

3.4 Simple Packet Block (optional)

The Simple Packet Block is a lightweight container for storing the packets coming from the network. Its presence is optional.

A Simple Packet Block is similar to a Packet Block (see Section 3.3Packet Block (optional)), but it is smaller, simpler to process and contains only a minimal set of information. This block is preferred to the standard Packet Block when performance or space occupation are critical factors, such as in sustained traffic dump applications. A capture file can contain both Packet Blocks and Simple Packet Blocks: for example, a capture tool could switch from Packet Blocks to Simple Packet Blocks when the hardware resources become critical.

The Simple Packet Block does not contain the Interface ID field. Therefore, it must be assumed that all the Simple Packet Blocks have been captured on the interface previously specified in the Interface Description Block.

Figure 6Simple Packet Block format. shows the format of the Simple Packet Block.



    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Packet Len                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   /                                                               /
   /                          Packet Data                          /
   /                      /* variable length */                    /
   /                                                               /
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 
 Simple Packet Block format. 

The Packet Block has the following fields:

The Simple Packet Block does not contain the timestamp because this is often one of the most costly operations on PCs. Additionally, there are applications that do not require it; e.g. an Intrusion Detection System is interested in packets, not in their timestamp.

A Simple Packet Block cannot be present in a Section that has more than one interface because of the impossibility to refer to the correct one (it does not contain any Interface ID field).

The Simple Packet Block is very efficient in term of disk space: a snapshot whose length is 100 bytes requires only 16 bytes of overhead, which corresponds to an efficiency of more than 86%.

3.5 Name Resolution Block (optional)

The Name Resolution Block is used to support the correlation of numeric addresses (present in the captured packets) and their corresponding canonical names and it is optional. Having the literal names saved in the file, this prevents the need of a name resolution in a delayed time, when the association between names and addresses can be different from the one in use at capture time. Moreover, the Name Resolution Block avoids the need of issuing a lot of DNS requests every time the trace capture is opened, and allows to have name resolution also when reading the capture with a machine not connected to the network.

A Name Resolution Block is normally placed at the beginning of the file, but no assumptions can be taken about its position. Name Resolution Blocks can be added in a second time by tools that process the file, like network analyzers.

The format of the Name Resolution Block is shown in Figure 7Name Resolution Block format..



    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Record Type              |         Record Length         | 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   /                       Record Value                            /
   /          /* variable length, aligned to 32 bits */            /
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
             . . . other records . . .
   |  Record Type == end_of_recs   |  Record Length == 00          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   /                                                               / 
   /                      Options (variable)                       / 
   /                                                               / 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 
 Name Resolution Block format. 

A Name Resolution Block is a zero-terminated list of records (in the TLV format), each of which contains an association between a network address and a name. There are three possible types of records:

Name Code Length Description
nres_endofrecord 0 0 It delimits the end of name resolution records. This record is needed to determine when the list of name resolution records has ended and some options (if any) begin.
nres_ip4record 1 Variable Specifies an IPv4 address (contained in the first 4 bytes), followed by one or more zero-terminated strings containing the DNS entries for that address.
nres_ip6record 2 Variable Specifies an IPv6 address (contained in the first 16 bytes), followed by one or more zero-terminated strings containing the DNS entries for that address.

Each Record Value is aligned to a 32-bit boundary. The corresponding Record Length reflects the actual length of the Record Value.

After the list of Name Resolution Records, optionally, a list of options (formatted according to the rules defined in Section 4Options) can be present.

In addiction to the options defined in Section 4Options, the following options are valid within this block:

Name Code Length Description
ns_dnsname 2 Variable A UTF-8 string containing the name of the machine (DNS server) used to perform the name resolution.
ns_dnsIP4addr 3 4 The IPv4 address of the DNS server.
ns_dnsIP6addr 4 16 The IPv6 address of the DNS server.

3.6 Interface Statistics Block (optional)

The Interface Statistics Block contains the capture statistics for a given interface and it is optional. The statistics are referred to the interface defined in the current Section identified by the Interface ID field.

The format of the Interface Statistics Block is shown in Figure 8Interface Statistics Block format..



    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Interface ID         |           Reserved            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Timestamp (High)                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Timestamp (Low)                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   /                                                               / 
   /                      Options (variable)                       / 
   /                                                               / 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 
 Interface Statistics Block format. 

The fields have the following meaning:

All the statistic fields are defined as options in order to deal with systems that do not have a complete set of statistics. Therefore, In addiction to the options defined in Section 4Options, the following options are valid within this block:

Name Code Length Description
isb_starttime 2 8 Time in which the capture started; time will be stored in two blocks of four bytes each, containing the timestamp in seconds and nanoseconds.
isb_endtime 3 8 Time in which the capture ended; time will be stored in two blocks of four bytes each, containing the timestamp in seconds and nanoseconds.
isb_ifrecv 4 8 Number of packets received from the physical interface starting from the beginning of the capture.
isb_ifdrop 5 8 Number of packets dropped by the interface due to lack of resources starting from the beginning of the capture.
isb_filteraccept 6 8 Number of packets accepted by filter starting from the beginning of the capture.
isb_osdrop 7 8 Number of packets dropped by the operating system starting from the beginning of the capture.
isb_usrdeliv 8 8 Number of packets delivered to the user starting from the beginning of the capture. The value contained in this field can be different from the value 'isb_filteraccept - isb_osdrop' because some packets could still lay in the OS buffers when the capture ended.

All the fields that refer to packet counters are 64-bit values, represented with the byte order of the current section. Special care must be taken in accessing these fields: since all the blocks are aligned to a 32-bit boundary, such fields are not guaranteed to be aligned on a 64-bit boundary.



 TOC 

4. Options

Almost all blocks have the possibility to embed optional fields. Optional fields can be used to insert some information that may be useful when reading data, but that it is not really needed for packet processing. Therefore, each tool can be either read the content of the optional fields (if any), or skip them at once.

Skipping all the optional fields at once is straightforward because most of the blocks are made of a first part with fixed format, and a second optional part. Therefore, the Block Length field (present in the General Block Structure, see Section 2.1General Block Structure) can be used to skip everything till the next block.

Options are a list of Type - Length - Value fields, each one containing a single value:

Options may be repeated several times (e.g. an interface that has several IP addresses associated to it). The option list is terminated by a special code which is the 'End of Option'.

The format of the optional fields is shown in Figure 9Options format..



    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Option Code              |         Option Length         | 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   /                       Option Value                            /
   /          /* variable length, aligned to 32 bits */            /
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   /                                                               /
   /                 . . . other options . . .                     /
   /                                                               /
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Option Code == opt_endofopt  |  Option Length == 0          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 
 Options format. 

The following codes can always be present in any optional field:

Name Code Length Description
opt_endofopt 0 0 It delimits the end of the optional fields. This block cannot be repeated within a given list of options.
opt_comment 1 variable A UTF-8 string containing a comment that is associated to the current block.


 TOC 

5. Experimental Blocks (deserved to a further investigation)

5.1 Other Packet Blocks (experimental)

Can some other packet blocks (besides the two described in the previous paragraphs) be useful?

5.2 Compression Block (experimental)

The Compression Block is optional. A file can contain an arbitrary number of these blocks. A Compression Block, as the name says, is used to store compressed data. Its format is shown in Figure 10Compression Block format..



    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Compr. Type  |                                               |
   +-+-+-+-+-+-+-+-+                                               |
   |                                                               |
   |                       Compressed Data                         |
   |                                                               |
   |              /* variable length, byte-aligned */              |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 
 Compression Block format. 

The fields have the following meaning:

5.3 Encryption Block (experimental)

The Encryption Block is optional. A file can contain an arbitrary number of these blocks. An Encryption Block is used to store encrypted data. Its format is shown in Figure 11Encryption Block format..



    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Encr. Type  |                                               |
   +-+-+-+-+-+-+-+-+                                               |
   |                                                               |
   |                       Encrypted Data                          |
   |                                                               |
   |              /* variable length, byte-aligned */              |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 
 Encryption Block format. 

The fields have the following meaning:

5.4 Fixed Length Block (experimental)

The Fixed Length Block is optional. A file can contain an arbitrary number of these blocks. A Fixed Length Block can be used to optimize the access to the file. Its format is shown in Figure 12Fixed Length Block format.. A Fixed Length Block stores records with constant size. It contains a set of Blocks (normally Packet Blocks or Simple Packet Blocks), of wihich it specifies the size. Knowing this size a priori helps to scan the file and to load some portions of it without truncating a block, and is particularly useful with cell-based networks like ATM.



    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Cell Size            |                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
   |                                                               |
   |                        Fixed Size Data                        |
   |                                                               |
   |              /* variable length, byte-aligned */              |
   |                                                               |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 
 Fixed Length Block format. 

The fields have the following meaning:

5.5 Directory Block (experimental)

If present, this block contains the following information:

A directory block must be followed by at least N packets, otherwise it must be considered invalid. It can be used to efficiently load portions of the file to memory and to support operations on memory mapped files. This block can be added by tools like network analyzers as a consequence of file processing.

5.6 Traffic Statistics and Monitoring Blocks (experimental)

One or more blocks could be defined to contain network statistics or traffic monitoring information. They could be use to store data collected from RMON or Netflow probes, or from other network monitoring tools.

5.7 Event/Security Block (experimental)

This block could be used to store events. Events could contain generic information (for example network load over 50%, server down...) or security alerts. An event could be:



 TOC 

6. Conclusions

The file format proposed in this document should be very versatile and satisfy a wide range of applications. In the simplest case, it can contain a raw dump of the network data, made of a series of Simple Packet Blocks. In the most complex case, it can be used as a repository for heterogeneous information. In every case, the file remains easy to parse and an application can always skip the data it is not interested in; at the same time, different applications can share the file, and each of them can benfit of the information produced by the others. Two or more files can be concatenated obtaining another valid file.



 TOC 

7. Most important open issues



 TOC 

Appendix A. Packet Block Flags Word

The Packet Block Flags Word is a 32-bit value that contains link-layer inforation about the packet.

The meaning of the bits is the following:

Bit Number Description
0-1 Inbound / Outbound packet (00 = information not available, 01 = inbound, 10 = outbound)
2-4 Reception type (000 = not specified, 001 = unicast, 010 = multicast, 011 = broadcast, 100 = promiscuous).
5-8 FCS length, in bytes (0000 if this information is not available). This value overrides the if_fcslen option of the Interface Description Block, and is used with those link layers (e.g. PPP) where the length of the FCS can change during time.
9-15 Reserved.
16-31 link-layer-dependent errors (Bit 31 = symbol error, Bit 30 = preamble error, Bit 29 = Start Frame Delimiter error, Bit 28 = unaligned frame error, Bit 27 = wrong Inter Frame Gap error, Bit 26 = packet too short error, Bit 25 = packet too long error, Bit 24 = CRC error, other?? are 16 bit enough?).


 TOC 

Intellectual Property Statement

Full Copyright Statement

Acknowledgment