Circbuf

Overview

Elphel cameras use FPGA to compress images and video, the compressed frames are transferred from the FPGA to the system memory through pseudo-DMA channel implemented in the CPU (Axis Etrax FS). It is "pseudo" as physically the data still goes (over the system bus) from the FPGA to the CPU chip, buffered there in small chunks and then sent to the system memory. Overall on average it takes 5 bus cycles (100MHz clock) to transfer one 32-bit word (4 bytes) of data from the FPGA to the system memory - 80MB/sec. There are hardware provisions in the model 353 to support new feature of ETRAX FS (not available in earlier ETRAX 100LX used in Elphel earlier models 303,313,323 and 333 cameras) - allowing external device (FPGA in our case) to become the bus master. That can allow to increase the data rate to virtually full 400MB/sec (in large bursts), but no FPGA code is yet developed to support this mode.

Circular buffer in Elphel 353 camera

The compressed frames are stored in a large circular buffer, it is done similar in the previous cameras and with other compressor (Ogg Theora) but the following description applies to the current state of the firmware (7.1.0.1) of the model 353 camera with JPEG compressor (Theora branch is not ported yet as of September 2007). Currently buffer size - CCAM_DMA_SIZE is ~19MB as defined in include/asm-cris/elphel/c313a.h).

All DMA transfers are 32-bytes aligned (32 bytes is a size of the internal DMA buffer in Etrax), and the data received from the FPGA contains JPEG-encode bit stream -- everything between SOF and EOF tags, no header, all 0xff data bytes escaped by appropriate 0x00 codes as required by the standard, so no 2 0xff bytes can appear together - this fact is used to find out if the data in the circular buffer was overwritten by the newer image frames.

When FPGA outputs the whole frame (it does not know the frame size in advance, only after compression is over) it adds 12 more bytes (3 long words), aligning them to the end of the 32-byte chunk, adding zeros before the data if needed. The last 4 bytes contain a 24-bit byte length of the JPEG data with the high byte set to 0xff. 8 bytes before (2 long words) have timestamp of the frame exposure start - seconds from epoch and microseconds.

The following listing has a sample of the buffer data (generated by the /usr/local/bin/test_mmap:

4b7ff8:  000000a4  06000800  016c0120  0a3946c1  01ab010a  40404040  ffff0000  000378fe
000000: {d30b03cc} 8af1dc34  a5875e77  9d4cf334  204db683  b1533cc7  c59f949e  36b1af45
000008:  90e20102  5e9ad7f1  f68cd273  130a1dc5  7e50641c  41bd14ef  140720c5  73c0e02e
...
00de38:  20a50766  4e433fc9  7614f51c  087bd2e4  988ee64e  cdf41ca0  6af40e1c  0000003b
00de40:  00000000  00000000  00000000  00000000  00000000  46ff4a82  000303bd <ff0378fe>
00de48:  f6285e9f  aeacb0a5  00000000  00000000  00000000  46ff4a81  000a1930  ff037929
00de50: [00000000] 00000000  00000000  00000000  00000000  00000000  00000000  00000000

The buffer is circular, so before 0 there is 0x4b7fff - last long word in the buffer). In this example FPGA compressed frame starting from the very beginning of the circbuf, bit stream is 0x378fe bytes long (as written in the 0x00de47 - OR-ed with 0xff000000) so it ends in the long word 0xde3f : .. 1c 0e f4 6a 3b 00