xiAPI Image Data Flow¶

Overview¶

The image data that is delivered by the sensor is processed by multiple components of the data pipeline.
The first component is typically the Sensor Data Processor - FPGA.
Data is then transported over the communication link (USB, PCIe, Thunderbolt, FireWire) to the computer.
In the computer, the image data might be processed by Image Processing blocks.

Depending on the camera model - some blocks are bypassed or not implemented.

Camera Side¶

Sensor¶

The sensor data bit depth can be controlled using the SENSOR_DATA_BIT_DEPTH parameter. Decreasing this parameter can increase the maximum frame rate on some camera models.

Sensor Data Processor¶

Sensor Data Processor contains multiple blocks for shaping the data:

• FPN Correction
• Data Crop (for horizontal ROI)
  • WIDTH can be used to control Data Crop unit
• Digital Gain / Offset
  • GAIN_SELECTOR can be used on some cameras with value XI_GAIN_SELECTOR_ANALOG_ALL to select this data processor unit
• Lookup Table (LUT)
  • Lookup table can be controlled by parameters:
  • LUT_INDEX which select input index
  • LUT_VALUE which set output value for currently selected LUT index
  • LUT_ENABLE which enable or disable this module
• Data Packer
  • The number of bits per pixel can be controlled by parameter OUTPUT_DATA_BIT_DEPTH
  • Packing can be controlled by parameter OUTPUT_DATA_PACKING
    • If packing is disabled and number of bits per pixel is higher than 8 -every pixel is transported as 16 bits word. E.g. Active 10 bits of value is padded by 6 bits of zeros to get 16 bits word.
    • If packing is enabled - only the active bits are transported without being padded. E.g. Active 10 bits of pixel value is transported without additional padding.

Transport¶

Data is transported over the communication link (USB, PCIe, Thunderbolt, FireWire) to the computer.
Next, data is stored in computer RAM waiting for being processed or directly passed to the application.
The amount of transported data per second can be limited by the parameter LIMIT_BANDWIDTH.

Computer Side¶

On the computer side, the transported data can be delivered to GPU (see details) or CPU memory.
This can be selected by XI_PRM_TRANSPORT_DATA_TARGET.

If data is delivered into CPU memory, it might be processed by Image Processing blocks or can be handed over directly to the application bypassing the processing blocks.
By setting of XI_PRM_IMAGE_DATA_FORMAT = XI_FRM_TRANSPORT_DATA the computer data processing is completely disabled.
This could save CPU and RAM access resources for other purposes.

Image Processing¶

Image Processing contains multiple blocks:

• Unpacking - transforms packed data into unpacked (e.g. 10 bits per pixel packed data is converted into 16 bits per pixel for compatibility with next processing blocks).
• Auto Exposure Measurement - calculates the selected part of image characteristics to get input values for controlling exposure.
• Image Defects Correction - replaces the part of the image marked as defected by neighboring pixels.
• White Balance Measurement - calculates the average color of the image for correction of white balance executed later in Debayering block.
• Debayering - calculates output RGB image from the input pixels values organized in Bayer matrix.
• Crop and Shift - crops the image and changes output bit depth if needed.