Exporter Usage

How-To Export IPFIX

Each fixbuf application must have a single fbInfoModel_t instance that represents the Information Elements that the application understands. The fbInfoModelAlloc() call allocates a new Information Model with the IANA-managed information elements (current as of the fixbuf release date) preloaded. Additional vendor-specific information elements may be added with fbInfoModelAddElement(), fbInfoModelAddElementArray(), fbInfoModelReadXMLFile(), and fbInfoModelReadXMLData().

To create an Exporter, first create an fbSession_t attached to the application's fbInfoModel_t to hold the Exporter's Transport Session state using fbSessionAlloc(). If exporting via the Spread protocol, create an fbSpreadParams_t and set its session member to your newly defined fbSession_t, group names (a null terminated array), and Spread daemon name.

Then create an fbExporter_t to encapsulate the connection to the Collecting Process or the file on disk, using the fbExporterAllocFP(), fbExporterAllocFile(), fbExporterAllocNet(), fbExporterAllocBuffer(), or fbExporterAllocSpread() calls.

With an fbSession_t and an fbExporter_t available, create a buffer (fBuf_t) for writing via fBufAllocForExport().

Create and populate templates for addition to this session using fbTemplateAlloc() and fbTemplateAppendSpecArray(), fbTemplateAppendSpec(), or fbTemplateAppend()). The layout of the template usually matches the C struct that holds the record. Any gaps (due to alignment) in the C struct must be noted in both the data structure and the template.

Example code:

static fbInfoElementSpec_t exportTemplate[] = {
{"flowStartMilliseconds", 8, 0 },
{"flowEndMilliseconds", 8, 0 },
{"octetTotalCount", 8, 0 },
{"packetTotalCount", 8, 0 },
{"sourceIPv4Address", 4, 0 },
{"destinationIPv4Address", 4, 0 },
{"sourceTransportPort", 2, 0 },
{"destinationTransportPort", 2, 0 },
{"protocolIdentifier", 1, 0 },
{"paddingOctets", 3, 0 },
{"payload", 0, 0 },
};
typedef struct exportRecord_st {
uint64_t flowStartMilliseconds;
uint64_t flowEndMilliseconds;
uint64_t octetTotalCount;
uint64_t packetTotalCount;
uint32_t sourceIPv4Address;
uint32_t destinationIPv4Address;
uint16_t sourceTransportPort;
uint16_t destinationTransportPort;
uint8_t protocolIdentifier;
uint8_t padding[3];
fbVarfield_t payload;
} exportRecord_t;
fbTemplate_t *tmpl = fbTemplateAlloc(infoModel);
fbTemplateAppendSpecArray(tmpl, exportTemplate, UINT32_MAX, &err);
An IPFIX template or options template structure.
Definition private.h:208
gboolean fbTemplateAppendSpecArray(fbTemplate_t *tmpl, fbInfoElementSpec_t *spec, uint32_t flags, GError **err)
Appends information elements described by a specifier array to a template.
fbTemplate_t * fbTemplateAlloc(fbInfoModel_t *model)
Allocates a new empty template.
#define FB_IESPEC_NULL
Convenience macro defining a null information element specification initializer (fbInfoElementSpec_t)...
Definition public.h:1807
A single IPFIX Information Element specification.
Definition public.h:1815
A variable-length field value.
Definition public.h:1236

The flags member of fbInfoElementSpec_st allows creation of multiple templates from a single spec array. The template-building functions fbTemplateAppendSpec(), fbTemplateAppendSpecArray(), and fbTemplateContainsAllFlaggedElementsByName() take a parameter also named flags. If an fbInfoElementSpec_t's flags is 0, the spec is always used. When the spec's flags is non-zero, the spec is used only if spec's flags intersected with (bit-wise AND) the function's flags parameter equals the spec's flags. That is, the high bits of the flags parameter must include all the high bits of the spec's flags. In general, the flags member of an fbInfoElementSpec_t should have few high bits. The functions' flags parameter generally contains multiple high bits to include multiple specs. Consider this spec array:

fbInfoElementSpec_t spec_array[] = {
{"protocolIdentifier", 1, 0},
{"paddingOctets", 3, 8},
{"sourceTransportPort", 2, 1},
{"destinationTransportPort", 2, 1},
{"paddingOctets", 5, 16},
{"icmpTypeIPv4", 1, 2},
{"icmpCodeIPv4", 1, 2},
{"icmpTypeIPv6", 1, 4},
{"icmpCodeIPv6", 1, 4},
};

The protocol is included in every template since the flags member value is 0. The ports are included when the flags parameter is 1. When building a template that maps to the following C struct (where padding is necessary for member alignment), the flags parameter should be 9.

struct proto_ports = {
uint8_t protocol;
uint8_t padding[3];
uint16_t source_port;
uint16_t dest_port;
};

For ICMP records, the flags parameter should be 2 or 4 depending on whether the record is IPv4 or IPv6, respectively. To map the template to the following struct, the flags parameter should be 18 for IPv4 and 20 for IPv6.

struct proto_ports = {
uint8_t protocol;
uint8_t padding[5];
uint8_t icmp_type;
uint8_t icmp_code;
};

Add the templates to the session via fbSessionAddTemplate() or fbSessionAddTemplateWithMetadata().

If exporting via Spread, before calling fbSessionAddTemplate(), set the group that should receive this template with the fBufSetSpreadExportGroup() call. If more than 1 group should receive the template, use the fbSessionAddTemplatesMulticast() which will call fBufSetSpreadExportGroup() on the given group(s) multicast the template to the given group(s). For Spread, do not use fbSessionAddTemplate() to send to multiple groups.

Typically each template is added to the session twice, once as an internal template---which describes how the record appears in memory---and again as an external template---which describes how the record appears in the IPFIX message.

Templates use internal reference counting, so they may be added to multiple sessions, or to the same session using multiple template IDs or multiple domains, or as both an internal and an external template on the same session.

Once the templates have been added to the session, use fbSessionExportTemplates() to add the templates to the buffer and then set the internal and external template IDs with fBufSetInternalTemplate() and fBufSetExportTemplate(). You can then use fBufAppend() to write records into IPFIX Messages and Messages to the output stream.

If using Spread, call fBufSetSpreadExportGroup() to set the groups to export to on the buffer before calling fBufAppend().

By default, fBufAppend() will emit an IPFIX Message to the output stream when the end of the message buffer is reached on write. The fBufSetAutomaticMode() call can be used to modify this behavior, causing fBufAppend() to return FB_ERROR_EOM when at end of message. Use this if your application requires manual control of message export. In this case, fBufEmit() will emit a Message to the output stream. If your exporter was created via fbExporterAllocBuffer(), you may use fbExporterGetMsgLen() to get the message length.

If not in automatic mode and a session's templates do not fit into a single message, use fbSessionExportTemplate() to export each template individually instead of relying on fbSessionExportTemplates().

Manual control of message export is incompatible with template and information element metadata (fbSessionSetMetadataExportTemplates() and fbSessionSetMetadataExportElements()). There are several functions that may cause the metadata options records to be exported, and the session must be free to create a new record set or template set as needed.