Porting Theora to 353 cameras
Elphel model 333 cameras were able to encode and stream Ogg Theora with 6.3.9 firmware. However, this model is no longer produced by Elphel. The goal of this project is to use the full potential of 353 cameras (larger FPGA, faster CPU) in order to port the Ogg Theora feature on the new hardware. First, the camera should be able to upload small Theora clips via FTP.
Different solutions are possible as previously mentioned in the Theora page :
- Instantiate both the Theora compressor and the MJPEG compressor in the FPGA since the new Spartan 3E seems to be large enough to handle the two features.
- Replace the existing MJPEG compressor with the Theora compressor and implement software Theora to JPEG conversion if JPEG is needed.
- Replace the existing MJPEG compressor in the 353 camera with the Theora compressor and keep two configuration files on the camera filesystem (one for Theora the other for MJPEG). FPGA code could be changed at run-time according to the needs.
Dealing with the last solution seems to be a good start for this project as the other features mentioned can be added later. In all cases, both FPGA code and software code will need to be adapted.
Contents
Hardware updates
Compressor
Replace the MJPEG compressor in the 353 with the 333 Theora compressor (333 Theora FPGA code). Maybe some adaptations need to be done within the Theora compressor itself. However, assuming that the compressor worked in 333 cameras a simple instantiation should suffice.
Memory controller
Replace the 4-channel memory controller with an 8-channel memory interface as in the 333 model in order to handle Theora specific data.
| Channel | 333 Theora | 353 |
|---|---|---|
| 0 | 8/16 bit data from sensor to SDRAM | Data from the sensor (it can be combined with FPN correction data) to the SDRAM in either 8-bit mode or 16-bit |
| 1 | 16 bit data from SDRAM to FPN correction module | Reads FPN data from SDRAM (8bit - subtract, 8 bit - scale) and feeds it to FPGA for FPN correction of the data on the way from sensor to SDRAM (through channel 0) |
| 2 | 20x20 tiles of raw data from SDRAM to the color space converter and compressor | Provides data to JPEG encoder in 16x16x8bit MCUs. It can also provide data directly to CPU through DMA bypassing any compression |
| 3 | PIO read/write of SDRAM data | Provides bidirectional access from CPU through 8x32 bit window. |
| 4 | Writing current (restored) frame as YCbCr 4:2:0 data to SDRAM | |
| 5 | Reading previous frame as YCbCr 4:2:0 data from SDRAM | |
| 6 | Writing 12-bit pre-tokens to SDRAM (in macroblock-scan order, starting with AC[63] | |
| 7 | reading 12-bit pre-tokens from SDRAM in indexed coded order |
System Interface
Adapt the system interface to assure communication between FPGA and its environment.
A feature of the system interface is to establish communication between the system CPU and the different blocks within the FPGA (compressor, memory controller etc). This is done by selecting a block with the address bus a[12:0] connected to the internal address bus ial[7:0]:
x353.v always @ (posedge cwr) begin a0[7:0]<=ial[7:0]; // may want to change to just ia if ia will be late enough; wnr0 <= ~irnw; da_ctl0 <= (ial[7:0]==8'h00); // 0x00 WE to control 32-bit register (1 loc) da_dmamode0 <= (ial[7:0]==8'h01); // 0x01 select writing to dma_cntr/dma_raw (1 loc) da_sensormode0 <= (ial[7:0]==8'h02); // 0x02 select writing to sensorpix (1 loc) ... da_compressor0 <= (ial[7:1]==7'h06); // 0x0c..0x0d - will be farther decoded in the compressor module ...
Since memory controller and compressor are to be replaced these addresses have to be adapted.
353 system interface address table
| Address | da signal | Comments |
|---|---|---|
| 0x00 | da_ctl0 | WE to control 32-bit register (1 loc) |
| 0x01 | da_dmamode0 | select writing to dma_cntr/dma_raw (1 loc) |
| 0x02 | da_sensormode0 | select writing to sensorpix (1 loc) |
| 0x03 | da_virttrig0 | write virtual trigger threshold |
| 0x04 | da_sensortrig0 | select writing to sensorpix (1 loc) |
| 0x05 | da_sensortrig_lines0 | write number of lines to be transferred in a frame (or after trigger) |
| 0x07 | da_sens_dc0 | write to sensor DCDC converter frequency divider |
| 0x08 | da_dcm0 | tune SDRAM clock phase |
| 0x09 | da_saturation0 | write color saturation values |
| 0x0a | da_framesync_dly0 | write frame sync interrupt delay (in scan lines) |
| 0x0c..0x0d | da_compressor0 | will be farther decoded in the compressor module |
| 0x0e | da_table_a0 | write tables address |
| 0x0f | twr0 | write tables data |
| 0x1b | dcmrst | async signal to restart DCMs (if irnw=0) |
| 0x1c..0x1f | da_interrupts0 | interrupt control |
| 0x20..0x2f | da_dswe0 | select reading/writing to mcontr (16 locations) |
| 0x30 | da_mem0 | read/write to SDRAM buffer, autoincrement address |
| 0x40..0x43 | da_hist0 | write/read histogram related data/registers |
| 0x44..0x47 | da_rtc0 | real time clock |
| 0x48 | da_timestamp0 | write timestamp mode |
| 0x74 | da_xjtag0 | write to external (sensor board) JTAG |
| 0x78..0x7b | da_extsync0 | control of external sync module 0x78 - 0x7b |
333 system interface address table
| Address | da signal | Comments |
|---|---|---|
| 0x00 | da_ctl0 | WE to control 32-bit register (1 loc) |
| 0x01 | da_dmamode0 | select writing to dma_cntr/dma_raw (1 loc) |
| 0x02 | da_sensormode0 | select writing to sensorpix (1 loc) |
| 0x03 | da_virttrig0 | write virtual trigger threshold |
| 0x04 | da_sensortrig0 | select writing to sensorpix (1 loc) |
| 0x05 | da_sensortrig_lines0 | write number of lines to be transferred in a frame (or aftre trigger) |
| 0x06 | da_interrupts0 | interrupt mask |
| 0x07 | da_sens_dc0 | write to sensor DCDC converter frequency divider |
| 0x08 | da_dcm0 | tune SDRAM clock phase |
| 0x10 | da_debug | write 0x10 any - load debug shift, read (through cs4) -shift |
| 0x20..0x27 | da_mcontr0 | select reading/writing to mcontr (8 locations) |
| 0x28..0x2f | da_compressor0 | will be farther decoded in the compressor module |
| 0x2c..0x2d | da_table0 | write gamma tables address (same as in compressor) |
| 0x30 | da_mem0 | read/write to SDRAM buffer, autoincrement address |
Interrupts
Adapt the Theora compressor interrupts to the 353 interrupt vector .
The 353 interrupt vector can handle up to 16 different irq. However, only irq 0 to 8 are used :
x353.v
assign irq_in= {8'b0, // extra
compressor_done_compress, // 7 - will go high after all data is sent out (reset by compressor)
compressor_done_input, // 6 - will go high after EOT and persist until DCT is enabled (reset by compressor)
dcc_rdy, // 5 - obsolete in 333 - channel3 has 128 more dc coefficients (or some if the compression is finished)
compressor_eot, // 4 - compressor read in the last MCU (predictable time to end of transfer)
sr_sensortrig[2], // 3 - (sync to pclk) - level, reset by writing to sensor triggering command register
xfer_over_irq, // 2 - (sync to pclk) - frame acquisition over
trig_irq, // 1 - (sync to pclk) - external trigger
vacts}; // 0 - (sync to pclk) - frame sync
353 MJPEG compressor triggers irqs 4, 6 and 7. 333 Theora compressor is the source of two irqs :
compressor_all.v done, // single-cycle pulse when the compressed frame is sent out (IRQ source). reset_compress, // reset compressor done interrupt (writing any compressor command)
Adaptation should consist of assigning Theora compressor irq sources to vectors 6 and 7 for example if the MJPEG compressor is to be replaced.
DMA FIFOs
353 design uses 2 DMA FIFOs between the compressor and the system interface instead of one for the 333 Theora compressor. The two DMA FIFOs have also slight differences between the two design. Maybe a single 333 FIFO should be kept for the final design (?)
Others
- Looks like Theora compressor requires 90° phase clock that was abandoned in the 353 design.
- Is it necessary to keep histogram and timestamp blocks in a design which supports Theora only ? If not, some code will have to be removed from sysinterface and mcontr.
- ...?
Software updates
- Adapt FPGA definitions (x353.h) and most likely some other FPGA drivers.
- ...?
These lists of update are obviously incomplete. They need to be more detailed in order to make an exhaustive list of the changes that have to be done.
