Connection-less Collector

How-To Use libfixbuf with a Data Buffer

One may use libfixbuf to read IPFIX records from a data buffer (a uint8_t array or octet array), independent of the transport mdoe. The application must handle all connections and reading, and it passes libfixbuf the octet array to be decoded. The array must contain valid IPFIX and should begin with the standard IPFIX header. Ideally, the application should provide the necessary Templates before any data records to ensure that the application can decode all of the data records.

To use libfixbuf independent of the transport mode, the initial steps are the same as previously described:

The application creates an fbInfoModel_t using fbInfoModelAlloc() and adds any additional, vendor-specific information elements using fbInfoModelAddElement(), fbInfoModelAddElementArray(), fbInfoModelReadXMLFile(), or fbInfoModelReadXMLData().

It creates an fbSession_t using fbSessionAlloc(). For each record to be read, it allocates a fbTemplate_t (fbTemplateAlloc()) and specifies its information elements (fbTemplateAppendSpecArray(), fbTemplateAppendSpec(), or fbTemplateAppend()) and adds each Template as internal to the Session via fbSessionAddTemplate().

The application should NOT create an fbCollector_t. To create the fBuf_t, use fBufAllocForCollection() and set the second parameter to NULL. The application then has everything needed to start reading from the IPFIX source.

Ideally, the application will read the first 4 bytes of the message first to determine the length of the next IPFIX message. The first 2 bytes are the IPFIX version (0x000A) and the third and fourth bytes are the length of the IPFIX message (including these 4 initial bytes). The application should then continue reading the length of the IPFIX message into an allocated octet array. The octet array should then be set on the fBuf by calling fBufSetBuffer().

The application calls fBufNext() repeatedly to receive the data records until fBufNext() returns FALSE with error code FB_ERROR_BUFSZ. This error notifies the application that there is not enough data in the octet array to read a full IPFIX message. However, if the fBuf is in manual mode (see fBufSetAutomaticNextMessage()) AND the size of the octet array was the IFIX message length, fixbuf will first return an FB_ERROR_EOM which indicates the application should perform another read (if the application ignores FB_ERROR_EOM errors and calls fBufNext(), fBufNext() will then return FB_ERROR_BUFSZ).

If the application octet array was the size of the IPFIX message, the entire octet array should have been processed. However, if the octet array was larger than the IPFIX message length, additional data may remain in the octet array that belongs to the next IPFIX message. To determine how much unprocessed data remains in the octet array, use fBufRemaining(). Any remaining data should be copied to the beginning of the octet array and the remaining IPFIX message data should be read. After each read, the application needs to call fBufSetBuffer(). Note that fBufSetBuffer() sets the collector and exporter on the fBuf to NULL. The application should clear the FB_ERROR_BUFSZ and/or FB_ERROR_EOM error when they occur using g_clear_error().

fixbuf may return the following error codes if it encounters one of the below issues. The application should determine the error and respond appropriately.

• FB_ERROR_IPFIX
  • If the first 2 bytes != 0x000A
  • If the length in the header < 16
• FB_ERROR_EOM
  • If the application read only the message length and the fBuf is in manual mode). This means the remaining buffer length == 0 and the application should clear the error and perform another read
• FB_ERROR_BUFSZ
  • If the header message length > the given buffer length
  • if the given buffer == NULL
  • If the given buffer length < 16
  • If buffer length == 0

Example usage:

FILE *fp;
uint8_t buf[65535];
...
while (fread(buf, 1, 4, fp) == 4) {
   len = ntohs(*((uint16_t *)(buf+2)));
   rc = fread(buf+4, 1, len-4, fp);
   if (rc > 0) {
       fBufSetBuffer(fbuf, buf, rc+4);
   } else if (feof(fp))
   ....
   for (;;) {
       ret = fBufNext(fbuf, (uint8_t *)rec, &len, &err);
       if (FALSE == ret) {
          if (g_error_matches(err, FB_ERROR_DOMAIN, FB_ERROR_BUFSZ)){
             rem = fBufRemaining(fbuf);
             g_clear_error(&err);
             break;
          }
       }
    }
}

Previous: sFlow Collectors | Next: Lists in IPFIX