ElphelWiki - User contributions [en]

SCINI: Submersible Capable of under Ice Navigation and Imaging

2008-02-10T03:15:08Z

Bnewbold:

A group at the [http://www.mlml.calstate.edu Moss Landing Marine Labs] in Moss Landing, CA (USA) is working on a new submersible ROV (remotely operated vehicle) which will be first deployed in Antarctica in the fall of 2007; it is a three year development program ending in 2009.

They are planning on using an Elphel 353 video camera for the main navigation camera, and possibly adding on a second high resolution still camera sensor board for image collection. The ability to run video over ethernet with the rest of the communications means the 1000 foot (330 meter) tether does not need dedicated conductors for video transmission, which saves a lot of hassle! The configurability of exposure, color balance, frame rate, etc, through open standards and software means less reverse engineering and easier integration with existing robotics control software.

<center>[[Image:Camera_module_sm.jpg]]

Prototype Waterproof Camera Bottle</center>

Here's a first fisheye [http://mit.edu/bnewbold/scini/from_crary310.jpg image] from McMurdo station in Antarctica... not well focused, window reflections, bad exposure, blah blah. The black shapes around the edge are LED casings. We will try to post more stills and video clips when we start flying the ROV in the next week or two, bandwidth permitting.

It's beautiful here. --[[User:Bnewbold|Bnewbold]] 04:41, 10 October 2007 (CDT)

For more information about the project including a blog see http://scini.mlml.calstate.edu/

[[User:Bnewbold]] was the main contact on this wiki for the SCINI Project; see his talk page for Elphel-specific questions and software related to the SCINI Project.

Design Ideas

2007-11-23T22:16:07Z

Bnewbold: /* Design Ideas */

== Design Ideas ==
=== FPGA Theora Encoder for Videoconferencing ===
In 2005 Elphel implemented a subset of Theora video encoder in Xilinx® FPGA that is part of Elphel model 333 camera capable of compressing 1280x1024@30fps ([http://www.xilinx.com/publications/xcellonline/xcell_53/xc_pdf/xc_video53.pdf], [http://www.linuxdevices.com/articles/AT3888835064.html]), but the CPU in the camera was not fast enough for the job even when the hard part was made by the hardware. In the model 333 camera software was responsible for generating frame headers and Ogg encapsulation of the Theora bitstream provided by FPGA.
Knowing that [http://developer.axix.com Axis Communications AB] were going to release a new faster processor we decided to wait for it before proceeding with Theora in the camera and used plain old Motion JPEG for a while.

Now we have the the new [[Roadmap#Update_on_353.2F363_cameras|Model 353 camera]] tested and released to production - the camera that has a brand new [http://en.wikipedia.org/wiki/ETRAX_CRIS#ETRAX_FS ETRAX FS], more memory and larger FPGA and is already tested in JPEG mode. So now it is a perfect time to resurrect Theora code in the camera and move forward.

Current [http://elphel.cvs.sourceforge.net/elphel/camera333/fpga/x333 FPGA implementation] supports only INTRA and INTER NOMV frames - the goal was to provide efficient compression for the scenes where the camera does not move (CCTV, videoconferencing) and large part of the frame stays the same. To reduce the bandwidth more we need to utilize selective block encoding so if camera is looking at an empty hallway there would be no bitstream at all but INTRA frames - just header telling that no block was encoded.

Such ability to selectively encode blocks is already in the FPGA code but we never used it with the slow CPU - encoded block map is a part of the frame header and the header is built by software, currently - before the video starts. To move farther we need either add FPGA code to generate frame headers or make use of the faster processor and do it in software.

Such project requires both FPGA code development (we use, and the rest of the code is written in Verilog HDL) and driver/application code (usually in C). When I was writing code (and debugging it) for the original encoder of the 333 camera I had to do both, but it would be nice to make such development in a team.

''There are many project ideas for [http://code.google.com/soc Google SoC] about [http://theora.org Ogg Theora] on [http://xiph.org Xiph Foundation] [http://wiki.xiph.org/index.php/Summer_of_Code Wiki page]''
=== AJAX Camera Interface with PHP/fastCGI ===
In March of the 2006 I published an article in [http://linuxdevices.com LinuxDevices] - [http://linuxdevices.com/articles/AT5951285077.html "AJAX, LAMP, and liveDVD for a Linux-based camera"]. It was a result of an interesting project to create a camera web interface with sliders, semi-transparent overlays, embedded MPlayer video plugin and other fancy features. It would be nice if you could create something like that with regular web development tools, but it that case I had to cheat - modify (and create new) CGI programs (mostly compiled programs in C, some - shell scripts) running in the camera providing the server part of AJAX.

In the [[Roadmap#Update_on_353.2F363_cameras|Model 353 camera]] there is more memory (64MB system, 64 MB video buffer and 128MB flash) and the CPU is three times faster. This allows to expand the usage of the mainstream web development tools in the camera - replace binary CGI programs running on the server (in the camera) with the [http://www.php.net/ PHP] code - it seem to run nicely in the [http://www.fastcgi.com fastCGI] mode in the camera with [http://www.lighttpd.net lightTPD] web server.

We plan to update the driver interface to simplify hardware interfacing with the PHP code (i.e. replace IOCTL with read/writes) and unleash creativity of the web developers. Instead of having API to the hardware as something given (in some cases - even "taken", not "given" - when the API is unpublished and has to be reverse-engineered ) you'll be able to create one of you dream.

Or - of the dream of [http://www.dvinfo.net/conf/ Digital Video enthusiasts]? See:
* [[HD_cinema_camera_Development_FAQ]]
* [http://dvinfo.net/conf/showthread.php?t=63677 High Definition with Elphel model 333 camera]
* [http://dvinfo.net/conf/showthread.php?t=83044 Elphel 333 (practical thread)]
* [http://dvinfo.net/conf/showthread.php?t=85568 Elphel 333 HTML]

=== LAMP-based DVR ===
There is not much use of the network video camera if the stream is not recorded somewhere and attaching a hard drive to the camera is not always the best solution - disk storage would increase the size of the camera, camera could be mounted outdoors, there could be a requirement that the data should survive the destruction of the camera and so on. Obvious solution for that is to use off-camera digital video recorder (DVR) that shares the same LAN with the camera. Or better yet - a cluster of cameras so it can record and play back video from multiple cameras.

When developing control software for the camera ([http://linuxdevices.com/articles/AT5951285077.html AJAX, LAMP, and liveDVD for a Linux-based camera]) I used a prototype DVR with very basic features implemented using LAMP technology - [http://linuxdevices.com/files/misc/elphel_dvr_on_lamp.gif Camera, client, and the DVR]. To avoid problems with cross-domain scripts (servers in the camera and DVR) I used a small trick - while video from the DVR was coming to the client directly, all control commands (low bandwidth) went through the camera used as a proxy.

The production software should rather be DVR-centric and support multiple cameras. It should organize video records and be able to serve requested ones with specified resolution and video format, transcoding from the Ogg+MJPEG/Ogg+Theora used for recording camera streams using [http://en.wikipedia.org/wiki/MEncoder MEncoder] or a similar application.

It would also be very useful to have capability of live trascoding of several (CPU power permitting) videostreams being recorded for remote monitoring.

Another idea - make the DVR+cameras cluster look like (have interface of) several lower resolution cameras with one of established APIs so the unmodified 3-rd party CCTV software could be used to control Elphel high-resolution cameras.

=== Electronic Rolling Shutter Distortion Compensation ===
Most of the available CMOS image sensors use [[Electronic Rolling Shutter]] and the different lines of an image are exposed at different times. That leads to distortions that are most visible when the camera moves or rotates - i.e. the vertical objects will look tilted when filmed sideways from the moving car or during panning. Fast moving objects are also distorted, but the moving camera effect is more annoying. Because of this effect camera manufacturers are avoiding this class of otherwise high performance and inexpensive the sensors and use interline CCDs with true snapshot electronic shutters. Being able to compensate the distortion of ERS effect would make it possible to build high resolution, high frame rate and still inexpensive video cameras.

To some extent the effect of moving camera can be compensated by post-processing of the video, estimating the movement of the camera by comparing consecutive frames and assuming that the accelerations (changes in camera movement/rotation speed) during a single frame were low. This seem to be implemented in [http://www.guthspot.se/video/deshaker.htm#rolling%20shutter%20setting Deshaker for VirtualDub]

The quality of correction could be higher if the movement of the camera was tracked with higher temporal resolution. There are other applications that require high resolution (and precision) imagery that could benefit from such system. This can be tested by converting a camera (or one of the several sensors of the same camera - see [[10353#10359|sensor multiplexer board]]) into an "optical mouse". Regular optical mice have tiny cameras (with usually just 16x16 or 32x32 pixel resolution) running at a high frame rate and calculating correlation between images. Similar could be done with the Elphel reconfigurable hardware - run a small window on a regular sensor board and port/implement correlation code in the FPGA.

Supplementing recorded images/video from the main sensor with precise orientation/position of the camera during each line exposure will allow correction of the ERS distortion during post-processing.

=== Demosaic Algorithms in FPGA ===
What is it and why it is needed in [http://en.wikipedia.org/wiki/Demosaicing Demosaicing] in Wikipedia.
In Elphel cameras Bayer-encoded pixels are processed just in front of the compressor (JPEG/MJPEG, Ogg Theora). These compressors use 16x16 blocks of pixels converted to YCbCr (intensity and 2 color components), currently we use 4:2:0 that means that color (chroma) components have twice less the spacial resolution (in each direction) than intensity (luma). So for each 4 input pixels compressor needs 4 Y (luma) pixels and one of each Cb and Cr (chroma) ones.

Our first cameras used very simple algorithm to calculate YCbCr from the Bayer pixels and for each pixel it needed just 3x3 block of neighbors. And as these neighbors were needed for the outer pixels in the 16x16 blocks, FPGA had to read larger (18x18) overlapping blocks from the external memory (internal FPGA memory is much smaller and can not hold the whole image). In the later FPGA code 20x20 blocks are read in (to make possible implementation of the fancier demosaic algorithms), but that was not done - the outer pixels are discarded and still only 3x3 used.

There are several algorithms that provide good results with less artifacts (see Wikipedia article) and these detailed descriptions:
* [http://scien.stanford.edu/class/psych221/projects/99/tingchen/algodep/vargra.html Variable Number of Gradients]
* [http://web.cecs.pdx.edu/~cklin/demosaic/ Pixel Grouping]

So just implement one in the FPGA code of the camera? Or adapt those ideas to use use 4:2:0 encoding and convert to direct Bayer->YCbCr conversion (not Bayer->RGB->YCbCr)
=== Stereo Vision for Robots ===
For the 353 series of cameras we had developed a [[10353#10359|multiplexer board]] that can accommodate several sensor boards. As these boards are connected to the same FPGA it is rather easy to achieve a complete synchronization of the two sensors - i.e. just by skipping clock pulses to the sensors until their output frame sync pulses will match. When a pair of sensors is mechanically aligned some stereo processing can be performed on a line-by-line basis, storing the intermediate results in attached SDRAM chip and then improving the results by combining 1-d correlation data from multiple lines.

=== Visual Processing with the FPGA ===
Many machine vision algorithms can't be run on large image sizes in real time because of simple parallel-izable initial preprocessing passes; some of these could probably be conjoined with the color balancing and compression passes the elphel FPGA takes. A lot of research and products are built on top of the OpenCV libraries (initially released by intel and optimized for their processors but now under a BSD license and developed for several platforms; [http://www.youtube.com/watch?v=SNW0_wx_eHU simple example]). If parts of the library were reoptimized to take advantage of the FPGA, existing higher level code should get better performance with minimal tweaking.

There are some interesting algorithms to enhance images beyond the usual gamma/contrast corrections; see the [http://www.ri.cmu.edu/projects/project_518.html robotics institute] and CMU and their spin off [http://www.shadowilluminator.org/ shadow illuminator] for examples.

Real-time artistic visual filters should also be possible at higher resolutions and/or frame rates than currently possible. A classic example is the waking life film which implemented ([http://www.hollywoodjesus.com/movie/waking_life/11.jpg example frame]) or the newer a scanner darkly ([http://www.apple.com/trailers/warner_independent_pictures/ascannerdarkly/large.html trailer]). These both had a lot of rotoscoping by hand, but good effects can be had automatically: gaussian blur, edge detect, blur more, and reduce color pallet to ~4-5.

Resources:
* [http://www.cs.colostate.edu/cameron/applications.html Cameron project]: Old attempt at doing visual filters on an FPGA (2002?)
* [http://www.intel.com/technology/computing/opencv/index.htm OpenCV] project at Intel: [http://en.wikipedia.org/wiki/OpenCV wikipedia article], [http://opencvlibrary.sourceforge.net/ wiki]

=== Gyro Image Stabilization ===
By integrating a 3-axis MEMS gyroscope (and/or accelerometers) into the developer board, the camera could try to shift exposure times to vibration/rotation free instants on a frame-by-frame case; this is apparently a techinque used by digital still cameras to reduce bluring. Or, the stability information could be encoded in every frame (exif?), so that good frames can be identified during post-processing. Or, exposure times could be shortened when rotation is detected to minimize bluriness at the expense of light information.

MEMS sensors are getting cheap and i'm sure there are tons of other projects that could make use of the information. Maybe this could be left to a third party USB or serial device?

File talk:354-11-view10353.jpeg

2007-11-07T12:00:51Z

Bnewbold:

looks awesome! --[[User:Bnewbold|Bnewbold]] 06:00, 7 November 2007 (CST)

Talk:SCINI: Submersible Capable of under Ice Navigation and Imaging

2007-10-18T02:39:43Z

Bnewbold: troubles with a 353

hehe...

cool to see you where :)

Drop Elphel's flag somewhere ;)

== troubles with a 353 ==

I swapped in the 12v DC-DC converter and the power supply side
of things seems to be working fine: yellow ethernet light comes on. But
during boot the linux kernel crashes. See the end for the serial boot output. This was
after I tried reflashing the device...

The second issue came when I tried to reflash the camera. When it powers up
and I hold down the reset button, the bootloader does start sending out some
kind of low level ethernet packet (can see them with packet sniffer), so I
should be able to get fresh firmware in there, right? It doesn't make any
nfs queries (kernel doesn't make it that far? or is that part of the
bootloader sequence?).

I downloaded and compiled the latest (7.1.2.0) firmware, but the netboot and
flashit programs don't work as documented... first, the etraxfsboot program
is called with debug level 5 ('-d 5'), but it's max is 4, so I changed that.
flashit claims to start booting ("We're doing a flash write, this may take
up to a few minutes...") even without the camera connected, while netboot
throws errors about the "cmdline" program and stops at fsboot complaining
about parameters.

Is there a new way to flash the cameras I haven't figured out? Do I need a
different version of etraxfsboot? I'm running on an "installed" version of
the liveDVD, but I tried it straight off the liveDVD as well.

ETRAX FS NAND boot loader
=========================
Rev 1, Jul 14 2007 19:49:44
Boot config: 0x00040000->0x40440000, len 0x00200000, boot @ 0x40440000
CPU revision: 0x00000020
Bootloader main at 0x38000ce2
Data end: 0x3800570c
Bss: 0x38008000
Heap: 0x38008000
Identifying nand chip...
maf_id: 0x00000020; dev_id: 0x000000f1
mtd->size == 0x08000000
scan_bbt(); mtd->size == 0x08000000
scan_bbt() ...01; mtd->size == 0x08000000
len == 0x00000100; and BBT_LEN_1 == 0x00000400
scan_bbt() ...02; mtd->size == 0x08000000
scan_bbt() ...03; mtd->size == 0x08000000
scan_bbt() ...1; mtd->size == 0x08000000
scan_bbt() ...2; mtd->size == 0x08000000
scan_bbt() search_read_bbts(); mtd->size == 0x08000000
search_read_bbts() search primary table; mtd->size == 0x08000000
search_bbt() ...1; mtd->size == 0x08000000
search_bbt() ...2; maxblocks: 0x00000004; mtd->size == 0x08000000
Block: 0x000003ff
Check, if we found a bbt for each requested chip
Bad block table found at page0x0000ffc0, version 0x00000001
search_read_bbts() search mirror table
search_bbt() ...1; mtd->size == 0x08000000
search_bbt() ...2; maxblocks: 0x00000004; mtd->size == 0x08000000
Block: 0x000003ff
Block: 0x000003fe
Check, if we found a bbt for each requested chip
Bad block table found at page0x0000ff80, version 0x00000001
scan_bbt() ...3
Done.
Chip identified... 3; NAND 128MiB 3,3V 8-bit
type: 0x00000004
flags: 0x000000c5
size: 0x08000000
erasesize: 0x00020000
oobblock: 0x00000800
oobsize: 0x00000040
ecctype: 0x00000002
eccsize: 0x00000100
Oob info:
useecc: 0x00000002
eccbytes: 0x00000018
eccpos: 0x00000028 0x00000029 0x0000002a 0x0000002b 0x0000002c 0x0000002d 0x0000002e 0x0000002f 0x00000030 0x00000031 0x00000032 0x00000033 0x00000034 0x00000035 0x00000036 0x00000037 0x00000038 0x00000039 0x0000003a 0x0000003b 0x0000003c 0x0000003d 0x0000003e 0x0000003f
Bootload in progress...
New block 0x00040000;len: 0x00200000;start: 0x00040000
New block 0x00060000;len: 0x001e0000;start: 0x00060000
New block 0x00080000;len: 0x001c0000;start: 0x00080000
New block 0x000a0000;len: 0x001a0000;start: 0x000a0000
New block 0x000c0000;len: 0x00180000;start: 0x000c0000
New block 0x000e0000;len: 0x00160000;start: 0x000e0000
New block 0x00100000;len: 0x00140000;start: 0x00100000
New block 0x00120000;len: 0x00120000;start: 0x00120000
New block 0x00140000;len: 0x00100000;start: 0x00140000
New block 0x00160000;len: 0x000e0000;start: 0x00160000
New block 0x00180000;len: 0x000c0000;start: 0x00180000
New block 0x001a0000;len: 0x000a0000;start: 0x001a0000
New block 0x001c0000;len: 0x00080000;start: 0x001c0000
New block 0x001e0000;len: 0x00060000;start: 0x001e0000
New block 0x00200000;len: 0x00040000;start: 0x00200000
New block 0x00220000;len: 0x00020000;start: 0x00220000
complete, status 0x00000000, loaded 0x00200000 bytes
Data in DRAM:
0x25f005b0 0x009cedff 0xbeef05b0
Booting...
Uncompressing Linux...
Done. Now booting the kernel.
Linux version 2.6.19 (larry@benq) (gcc version 3.2.1 Axis release R64/1.64) #4 Thu Jul 26 10:19:26 CST 2007
Setting up paging and the MMU.
Linux/CRISv32 port on ETRAX FS (C) 2003, 2004 Axis Communications AB
Built 1 zonelists. Total pages: 8160
Kernel command line: root=/dev/mtdblock3 init=/linuxrc rootfstype=jffs2
PID hash table entries: 256 (order: 8, 1024 bytes)
Dentry cache hash table entries: 8192 (order: 2, 32768 bytes)
Inode-cache hash table entries: 4096 (order: 1, 16384 bytes)
Memory: 55656k/65536k available (1539k kernel code, 9880k reserved, 510k data, 64k init)
Mount-cache hash table entries: 1024
NET: Registered protocol family 16
SCSI subsystem initialized
NET: Registered protocol family 2
IP route cache hash table entries: 512 (order: -2, 2048 bytes)
TCP established hash table entries: 2048 (order: 0, 8192 bytes)
TCP bind hash table entries: 1024 (order: -1, 4096 bytes)
TCP: Hash tables configured (established 2048 bind 1024)
TCP reno registered
Check hardware configuration of Elphel 353 camera
Elphel 10349 IO extension board is present
fast_timer_init()
JFFS2 version 2.2. (NAND) (C) 2001-2006 Red Hat, Inc.
io scheduler noop registered (default)
Serial: CRISv32 driver $Revision: 1.78 $ <6>ttyS0 at I/O 0xb0026000 (irq = 68) is a CRISv32
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
PPP generic driver version 2.4.2
ETRAX FS 10/100MBit ethernet v0.01 (c) 2003 Axis Communications AB
eth0: changed MAC to 00:40:8C:CD:00:00
ETRAX FS stream co-processor driver v0.01, (c) 2003 Axis Communications AB
stream co-processor: init done.
axisflashmap: ...
NAND device: Manufacturer ID: 0x20, Chip ID: 0xf1 (ST Micro NAND 128MiB 3,3V 8-bit)
NAND 128MiB 3,3V 8-bit: 0x08000000 bytes of NAND flash memory.
axisflashmap: Set to boot from NAND flash, making NAND flash primary device.
axisflashmap: flash read 512 bytes at 0x00040000, data: 00 b0 05 f0 25 ff ed 9c
axisflashmap: partition table offset 10, data: ef be 84 00 ac 0e 00 00
axisflashmap: Found a valid partition table at 0xc02339ae-0xc0233a3a.
offset == 0x00040000, size == 0x007C0000
offset == 0x00800000, size == 0x01800000
offset == 0x02000000, size == 0x05000000
offset == 0x07000000, size == 0x00C00000
Creating 1 MTD partitions on "NAND 128MiB 3,3V 8-bit":
0x00000000-0x08000000 : "main"
Creating 1 MTD partitions on "NAND 128MiB 3,3V 8-bit":
0x00000000-0x00040000 : "part0"
Creating 1 MTD partitions on "NAND 128MiB 3,3V 8-bit":
0x00040000-0x00800000 : "part1"
Creating 1 MTD partitions on "NAND 128MiB 3,3V 8-bit":
0x00800000-0x02000000 : "part2"
Creating 1 MTD partitions on "NAND 128MiB 3,3V 8-bit":
0x02000000-0x07000000 : "part3"
Creating 1 MTD partitions on "NAND 128MiB 3,3V 8-bit":
0x07000000-0x07c00000 : "part4"
ETRAX FS GPIO driver v2.5, (c) 2003-2006 Axis Communications AB
Elphel (R) model 353 FPGA (Xilinx (R) XC3S1200E) configuration driverElphel (R) model 353 system clocks (PLL frequency synth.) driverfpga_controlCMOS/MCP camera interrupts initialized
Initializing DMA registers for EXTDMA3 - CMOS/MCP camera 353 driver
Elphel (R) model 353 i2c character device driverhist: init ok
ip_conntrack version 2.4 (512 buckets, 4096 max) - 196 bytes per conntrack
ip_tables: (C) 2000-2006 Netfilter Core Team
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
Bridge firewalling registered
VFS: Mounted root (jffs2 filesystem) readonly.
Freeing unused kernel memory: 64k freed
exit: 4: Illegal number: X
/linuxrc: Mounting /mnt/flash... Empty flash at 0x00377bb8 ends at 0x00378000
done
/sbin/rc.fsupgrade: Cannot find /usr/etc/defaultfiles/jimage
/linuxrc: /sbin/rc.fsupgrade failed!
/linuxrc: Starting init.
init: starting up.
Starting system initialization.

Axis Linux, http://developer.axis.com/
Copyright (C) 2000-2007 Axis Communications AB

Elphel (R) Model 353 Camera release 7.1.0.9

�[32;01m*�[0m Mounting filesystems...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Initializing /var...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Determining MAC address...

�[A�[4C�[32;01m+�[0m using saved MAC address: 00:0E:64:09:00:13

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Initializing random number generator...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Setting hostname...

�[A�[4C�[32;01m+�[0m hostname: "Elphel353"

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Initializing volatile resolver configuration...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Starting ipsetd...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Setting FPGA master clock to 125 MHz...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Programming FPGA with /etc/x353.bit...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
CR[13]=1 (was 0) �[32;01m*�[0m Loaded FPGA rev...

�[2A�[72C�[34;01m[ �[32;01m3533010�[34;01m ]�[0m
�[32;01m*�[0m Adjusting DDR SDRAM clock phase...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Writing Huffman tables from /etc/huffman.dat...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Writing default JPEG header from /etc/header.jpeg...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
writing precalculated gamma tables from /etc/gamma.dat
�[32;01m*�[0m Turn on xtall output (12MHz) to FPGA...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Looking for sensor FPGA...

�[2A�[71C�[34;01m[ �[31;01mnone�[34;01m ]�[0m
1970-01-01 00:00:15: (mod_fastcgi.c.1289) --- fastcgi spawning local
proc: /bin/php-fcgi
port: 0
socket /tmp/php.socket
min-procs: 1
max-procs: 1
1970-01-01 00:00:15: (mod_fastcgi.c.1314) --- fastcgi spawning
port: 0
socket /tmp/php.socket
current: 0 / 1
�[32;01m*�[0m Everything in /etc/init.d/fpga is done...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Configure broadcast ping...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
System initialization is done.
/bin/sh: can't access tty; job control turned off
# 1970-01-01 00:00:15: (log.c.75) server started
New runlevel: 3
�[32;01m*�[0m Starting respawn daemon...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Starting system and kernel logger...
Jan 1 00:00:15 Elphel353 syslogd 1.4.1: restart.
�[A�[4C�[32;01m+�[0m system logger started

�[A�[4C�[32;01m+�[0m kernel logger started

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Bringing lo up...

�[2A�[72C�[34;01m[ �[32;01mok�[34;01m ]�[0m
�[32;01m*�[0m Bringing eth0 up...
eth0: changed MAC to 00:0E:64:09:00:13

�[A�[4C�[32;01m+�[0m hardware address: 00:0E:64:09:00:13

�[A�[4C�[32;01m+�[0m media: auto

�[A�[4C�[32;01m+�[0m eth0 going up
eth0: changed MAC to 00:0E:64:09:00:13

�[A�[4C�[32;01m+�[0m IPv4 on eth0

�[A�[4C�[32;01m+�[0m boot protocol: none
Unable to handle kernel NULL pointer dereference at virtual address 00000000
Oops: 0002
CPU: 0
ERP: c00e8ee2 SRP: c00164a8 CCS: 0a00a008 USP: afa8d588 MOF: 00000000
r0: c0b15000 r1: c0b15000 r2: c0b15260 r3: c0b15d79
r4: c01c9568 r5: c00164a8 r6: c01c956c r7: c0012b74
r8: 00000000 r9: 00200200 r10: c0b15000 r11: c0b15cd1
r12: c0b15ccd r13: c3c6dbdc oR10: c0b15000 acr: 00000000
sp: c3c6db44
Data MMU Cause: 000002d4
Instruction MMU Cause: 000180ff
Process ip (pid: 699, stackpage=c0baed80)

Stack from afa8d588:
Failing address 0xafa8d588

Call Trace: Failing address 0xafa8d588

Stack from c3c6da64:
c0004988 c3c6db44 c000eaae c3c6db70 00000000 00000000 c3c6db44 c0005672
00000000 c0baed80 c3c6db44 00000002 00000000 c0b15000 c0208200 00000000
c0b15000 c0b15000 c0b15260 c0b15d79 c01c9568 c00164a8 c01c956c c0012b74
Call Trace: [<c000eaae>] [<c0005672>] [<c00164a8>] [<c0012b74>] [<c00164a8>] [<c0012b74>] [<c00164a8>]
[<c00e8ee2>] [<c00e8ec4>] [<c0012bf2>] [<c0004bb4>] [<c000635c>] [<c002d150>] [<c00cb152>] [<c002d422>]
[<c002d50c>] [<c002d566>] [<c0034aa8>] [<c000556c>] [<c0034aa8>] [<c000556c>] [<c000a1ba>] [<c0032ace>]
[<c0004bb4>] [<c00061b8>] [<c0035904>] [<c003653e>] [<c00317be>] [<c00639e6>] [<c00cb4a1>] [<c0064d3a>]
[<c00436b4>] [<c0042730>] [<d7600009>] [<c004278a>] [<c004348c>] [<c004278a>] [<c0005926>] [<c00074c8>]
Code: 60 36 3f 3c 79 0d 63 fa 06 20 60 16 (7f) 8a 3f 1c e5 0c 41 b2 bf be 9e 03
Kernel panic - not syncing: Aiee, killing interrupt handler!

10349

2007-10-17T23:23:46Z

Bnewbold: /* Connector pinouts */ - ascii art pin layout diagram

Russia page [[10349.Ru]]

[[Image:10349RevC.jpg|thumb|Elphel 10349RevC boaard]]
[[Image:IDE-HD-1.8p.jpg|thumb|1.8" IDE HD with zif connector]]
[[Image:10349RevCon10353.jpg|thumb|10349 RevC mounted on 10353]]
=== Interface board. ===

* serial port
* 4 ports USB hub with 3 internal, 1 external USB ports
* IDE port for the 1.8" hard drive with ZIF Flat cable
* 2 socket for external usermade board
* some points for soldering wires directly to board (from external board to 10349 for example), easy hardware developing.

=== Connector pinouts ===

internal USB2, USB3, USB4
1 - power
2 - D-
3 - D+
4 - GND

J7 is the serial port
1 - RX
2 - TX
3 - CTS
4 - RTS
5 - GND
6 - removed, used as key

J7 pin layout (1 is marked)

(GND) (RX)
5 3 1
6 4 2
(TX)

J4 for connect external boards. All wires of J4 connected to the solder pads. For make connection via wires.

J5 for connect external boards
1 - EXT0 (FPGA IO signal)
2 - EXT1 (FPGA IO signal)
3 - EXT2 (FPGA IO signal)
4 - EXT3 (FPGA IO signal)
5 - EXT4 (FPGA IO signal)
6 - GND
7 - EXT5 (FPGA IO signal)
8 - GND
9 - +3.3V
10 - +3.3V
11 - CTS (TTL level)
12 - RTS (TTL level)
13 - RX (TTL level)
14 - TX (TTL level)
15 - not connected
16 - not connected

=== Power description ===

It has DC-DC converter 3.3V->5V 1.5A. Each USB port has current limit to 300mA. Each port is overcurrent protected via USB power distributed systems. Use both IDE hard drive with USB devices can be overload Power over Ethernet equipment. Use external additional power injector.

==== Power consumption in different configurations ====

Here are some results on power consumption test made in different configurations with Elphel 353 camera, 5MPix sensor, 10349 board, 1,8" HD and USB staff...

<table border="1">
<tr><td>2400 milliwatts</td><td>just booted</td></tr>
<tr><td>3000 milliwatts</td><td>after setting image parameters</td></tr>
<tr><td>3700 milliwatts</td><td>streamer on</td></tr>
<tr><td>3300 milliwatts</td><td>streamer off</td></tr>
<tr><td>3800 milliwatts</td><td>streamer on and somebody playing the stream</td></tr>
<tr><td>5000 milliwatts</td><td>with streamer on and HD writing at full speed (dd)</td></tr>
<tr><td>5800 milliwatts</td><td>streamer on, HD and USB-flash writing at full speed</td></tr>
<tr><td>3300 milliwatts</td><td>back to streamer off</td></tr>
</table>

Tests was performed with [http://www.linksys.com/servlet/Satellite?c=L_Product_C2&childpagename=US%2FLayout&cid=1130276637958&pagename=Linksys%2FCommon%2FVisitorWrapper&lid=3795822279B15 Linksys SRW224P] POE switch

=== Kit for Developers ===

* for serial cable - connector A3B-6D-2C (Digi-Key) - 1 + half of AE9873-ND (Digi-Key) - 1
* for internal USB sockets - AE10179-ND (Digi-Key) - 1
* for additional boards connectors 3M5311-ND (thru-hole 2mm, Digi-Key) - 2
* or for additional boards connectors 3M5309CT-ND (SMD, Digi-Key) - 2
* blank PCB board V1255-ND (Digi-Key) - 1

=== Next revision ===

In new revision of this board we want use much more IDE interfaces.

* IDE port for a 2.5" hard drive
* IDE port for CompactFlash card
* IDE port a 1.8" hard drive with ZIF Flat cable
* SATA port

For make SATA port we can use IDE<->SATA bridge or soft IP core in a FPGA.

IDE-SATA bridges:

* JMicron [[http://www.jmicron.com.tw/JM20330.html JM20330]] (send email, no answer)
* Marvell [[http://www.marvell.com/products/storage/sata/index.jsp 88i8030, 88SA8040]] (need sign NDA, sucks)
* Silicon Image [[http://www.siliconimage.com/products/product.aspx?id=30 SiI3611]] (Available only through your local sales contact. send email, no answer)
* Atmel [[http://www.atmel.com/dyn/products/product_card.asp?family_id=677&family_name=Data+Storage&part_id=3456 AT78C5010]] (Available only through your local sales contact. send email, no answer)
* ACARD [[http://www.acard.com/english/fb01-product.jsp?prod_no=ARC-770&type1_title=Storage%20IC&type1_idno=1&idno_no=60 ARC-770]] (Available only through your local sales contact. send email, no answer)

Now I can`t found any datasheets for this. These are big secrets of the companies :(

=== Body of camera for developers ===

Case A consist of a large window on the top and transparent plex panel for closing. Anybody can
easy modified this panel for any application.

Case B consist a large space with two bodies. Can use same as back pannel can use for backpanel plex too.

[[Image:Devbody-1.png|thumb|Body for developers kit A]]
[[Image:Devbody-2.png|thumb|Body for developers kit B]]

[[Image:Developer-body-1_1.png|thumb|Body for developers kit other of A]]
[[Image:Developer-body-2_1.png|thumb|Body for developers kit other of B]]

Talk:Using the cameras

2007-10-17T23:19:18Z

Bnewbold: serial baud rate

== Using the 323 with Flash Illumination -- Suggested settings ==

I photographed a GretagMacbeth Color Rendition Chart with a 323 camera equipped with a Grade 2 color microlens CCD. The lens was a 60mm MicroNikkor, set at wide open aperture. Illumination was from a single Elinchrom 600S set to about 25 watt seconds, equipped with a 14x20" Chimira softbox with both diffusing panels in place. With the target in a vertcal plane and the camera aligned on the center normal, the bottom of the softbox was positioned about 6" above the back of the 323, angled down at the target. The CCD to target distance was about 0.8m. The soft box to target was about 0.95m along the center to center beeline.

The following image is reasonably numerically (+- 5% typical) close to the RGB specifications for the target, and subjectively looks very good.
[[Image:121705-13.jpg]]

The chart is rotated to avoid the huge horizontal QE falloff of the CCD.

With a Minolta IVF light meter in non-corded mode, the (incident) illumination reads f/2.8 at ISO 32.

Screenshot of the main parameters page:
[[Image:121705-13 Screenshot.jpg]]

This camera uses a test version of software that allows capture to be triggered by pulling CTS1 low on the master compressor. This pin is available on the power supply header connector. The flash is triggered via Pocket Wizard, whose Port 1 input is paralleled to CTS1. Both inputs float at 3.3v. The exposure period must be set long at 2 ms because the Pocket Wizard radio link takes about 1 mS.

From this example, it appears that if you can nail the exposure, you'll get good photos. Until higher resolution raw data becomes available (preferably in [http://www.adobe.com/products/dng/main.html Adobe DNG format]), this is a good workaround.

== serial baud rate ==

using the 10394 board with a 353 camera, the correct baud rate for the serial port seems to be 115200, with version 7.1.0.9 firmware.

SCINI: Submersible Capable of under Ice Navigation and Imaging

2007-10-10T09:41:09Z

Bnewbold: added photo link

A group at the [http://www.mlml.calstate.edu Moss Landing Marine Labs] in Moss Landing, CA (USA) is working on a new submersible ROV (remotely operated vehicle) which will be first deployed in Antarctica in the fall of 2007; it is a three year development program ending in 2009.

They are planning on using an Elphel 353 video camera for the main navigation camera, and possibly adding on a second high resolution still camera sensor board for image collection. The ability to run video over ethernet with the rest of the communications means the 1000 foot (330 meter) tether does not need dedicated conductors for video transmission, which saves a lot of hassle! The configurability of exposure, color balance, frame rate, etc, through open standards and software means less reverse engineering and easier integration with existing robotics control software.

<center>[[Image:Camera_module_sm.jpg]]

Prototype Waterproof Camera Bottle</center>

Here's a first fisheye [http://mit.edu/bnewbold/scini/from_crary310.jpg image] from McMurdo station in Antarctica... not well focused, window reflections, bad exposure, blah blah. The black shapes around the edge are LED casings. We will try to post more stills and video clips when we start flying the ROV in the next week or two, bandwidth permitting.

It's beautiful here. --[[User:Bnewbold|Bnewbold]] 04:41, 10 October 2007 (CDT)

For more information about the project including a blog see http://scini.mlml.calstate.edu/

[[User:Bnewbold]] is the main contact on this wiki for the SCINI Project; see his talk page for Elphel-specific questions and software related to the SCINI Project.

SCINI: Submersible Capable of under Ice Navigation and Imaging

2007-10-10T09:37:31Z

Bnewbold:

A group at the [http://www.mlml.calstate.edu Moss Landing Marine Labs] in Moss Landing, CA (USA) is working on a new submersible ROV (remotely operated vehicle) which will be first deployed in Antarctica in the fall of 2007; it is a three year development program ending in 2009.

They are planning on using an Elphel 353 video camera for the main navigation camera, and possibly adding on a second high resolution still camera sensor board for image collection. The ability to run video over ethernet with the rest of the communications means the 1000 foot (330 meter) tether does not need dedicated conductors for video transmission, which saves a lot of hassle! The configurability of exposure, color balance, frame rate, etc, through open standards and software means less reverse engineering and easier integration with existing robotics control software.

<center>[[Image:Camera_module_sm.jpg]]

Prototype Waterproof Camera Bottle</center>

Here's a first fisheye image from McMurdo station in Antarctica: http://mit.edu/bnewbold/scini/from_crary310.jpg
Not well focused, window reflections, bad exposure, blah blah. The black shapes around the edge are LED casings.

For more information about the project including a blog see http://scini.mlml.calstate.edu/

[[User:Bnewbold]] is the main contact on this wiki for the SCINI Project; see his talk page for Elphel-specific questions and software related to the SCINI Project.

Talk:Images and videos examples

2007-09-06T04:39:39Z

Bnewbold: 3mpix question

Do the 3MPix images mean that the 1A2031A revA boards are save to use? [[User:Bnewbold|Bnewbold]] 23:39, 5 September 2007 (CDT)

Design Ideas

2007-08-16T05:38:35Z

Bnewbold: /* Visual Processing with the FPGA */

Design Ideas

2007-08-16T05:33:34Z

Bnewbold: /* Design Ideas */

== Design Ideas ==
=== FPGA Theora Encoder for Videoconferencing ===
In 2005 Elphel implemented a subset of Theora video encoder in Xilinx® FPGA that is part of Elphel model 333 camera capable of compressing 1280x1024@30fps ([http://www.xilinx.com/publications/xcellonline/xcell_53/xc_pdf/xc_video53.pdf], [http://www.linuxdevices.com/articles/AT3888835064.html]), but the CPU in the camera was not fast enough for the job even when the hard part was made by the hardware. In the model 333 camera software was responsible for generating frame headers and Ogg encapsulation of the Theora bitstream provided by FPGA.
Knowing that [http://developer.axix.com Axis Communications AB] were going to release a new faster processor we decided to wait for it before proceeding with Theora in the camera and used plain old Motion JPEG for a while.

Now we have the the new [[Roadmap#Update_on_353.2F363_cameras|Model 353 camera]] tested and released to production - the camera that has a brand new [http://en.wikipedia.org/wiki/ETRAX_CRIS#ETRAX_FS ETRAX FS], more memory and larger FPGA and is already tested in JPEG mode. So now it is a perfect time to resurrect Theora code in the camera and move forward.

Current [http://elphel.cvs.sourceforge.net/elphel/camera333/fpga/x333 FPGA implementation] supports only INTRA and INTER NOMV frames - the goal was to provide efficient compression for the scenes where the camera does not move (CCTV, videoconferencing) and large part of the frame stays the same. To reduce the bandwidth more we need to utilize selective block encoding so if camera is looking at an empty hallway there would be no bitstream at all but INTRA frames - just header telling that no block was encoded.

Such ability to selectively encode blocks is already in the FPGA code but we never used it with the slow CPU - encoded block map is a part of the frame header and the header is built by software, currently - before the video starts. To move farther we need either add FPGA code to generate frame headers or make use of the faster processor and do it in software.

Such project requires both FPGA code development (we use, and the rest of the code is written in Verilog HDL) and driver/application code (usually in C). When I was writing code (and debugging it) for the original encoder of the 333 camera I had to do both, but it would be nice to make such development in a team.

''There are many project ideas for [http://code.google.com/soc Google SoC] about [http://theora.org Ogg Theora] on [http://xiph.org Xiph Foundation] [http://wiki.xiph.org/index.php/Summer_of_Code Wiki page]''
=== AJAX Camera Interface with PHP/fastCGI ===
In March of the 2006 I published an article in [http://linuxdevices.com LinuxDevices] - [http://linuxdevices.com/articles/AT5951285077.html "AJAX, LAMP, and liveDVD for a Linux-based camera"]. It was a result of an interesting project to create a camera web interface with sliders, semi-transparent overlays, embedded MPlayer video plugin and other fancy features. It would be nice if you could create something like that with regular web development tools, but it that case I had to cheat - modify (and create new) CGI programs (mostly compiled programs in C, some - shell scripts) running in the camera providing the server part of AJAX.

In the [[Roadmap#Update_on_353.2F363_cameras|Model 353 camera]] there is more memory (64MB system, 64 MB video buffer and 128MB flash) and the CPU is three times faster. This allows to expand the usage of the mainstream web development tools in the camera - replace binary CGI programs running on the server (in the camera) with the [http://www.php.net/ PHP] code - it seem to run nicely in the [http://www.fastcgi.com fastCGI] mode in the camera with [http://www.lighttpd.net lightTPD] web server.

We plan to update the driver interface to simplify hardware interfacing with the PHP code (i.e. replace IOCTL with read/writes) and unleash creativity of the web developers. Instead of having API to the hardware as something given (in some cases - even "taken", not "given" - when the API is unpublished and has to be reverse-engineered ) you'll be able to create one of you dream.

Or - of the dream of [http://www.dvinfo.net/conf/ Digital Video enthusiasts]? See:
* [[HD_cinema_camera_Development_FAQ]]
* [http://dvinfo.net/conf/showthread.php?t=63677 High Definition with Elphel model 333 camera]
* [http://dvinfo.net/conf/showthread.php?t=83044 Elphel 333 (practical thread)]
* [http://dvinfo.net/conf/showthread.php?t=85568 Elphel 333 HTML]

=== LAMP-based DVR ===
There is not much use of the network video camera if the stream is not recorded somewhere and attaching a hard drive to the camera is not always the best solution - disk storage would increase the size of the camera, camera could be mounted outdoors, there could be a requirement that the data should survive the destruction of the camera and so on. Obvious solution for that is to use off-camera digital video recorder (DVR) that shares the same LAN with the camera. Or better yet - a cluster of cameras so it can record and play back video from multiple cameras.

When developing control software for the camera ([http://linuxdevices.com/articles/AT5951285077.html AJAX, LAMP, and liveDVD for a Linux-based camera]) I used a prototype DVR with very basic features implemented using LAMP technology - [http://linuxdevices.com/files/misc/elphel_dvr_on_lamp.gif Camera, client, and the DVR]. To avoid problems with cross-domain scripts (servers in the camera and DVR) I used a small trick - while video from the DVR was coming to the client directly, all control commands (low bandwidth) went through the camera used as a proxy.

The production software should rather be DVR-centric and support multiple cameras. It should organize video records and be able to serve requested ones with specified resolution and video format, transcoding from the Ogg+MJPEG/Ogg+Theora used for recording camera streams using [http://en.wikipedia.org/wiki/MEncoder MEncoder] or a similar application.

It would also be very useful to have capability of live trascoding of several (CPU power permitting) videostreams being recorded for remote monitoring.

Another idea - make the DVR+cameras cluster look like (have interface of) several lower resolution cameras with one of established APIs so the unmodified 3-rd party CCTV software could be used to control Elphel high-resolution cameras.

=== Electronic Rolling Shutter Distortion Compensation ===
Most of the available CMOS image sensors use [[Electronic Rolling Shutter]] and the different lines of an image are exposed at different times. That leads to distortions that are most visible when the camera moves or rotates - i.e. the vertical objects will look tilted when filmed sideways from the moving car or during panning. Fast moving objects are also distorted, but the moving camera effect is more annoying. Because of this effect camera manufacturers are avoiding this class of otherwise high performance and inexpensive the sensors and use interline CCDs with true snapshot electronic shutters. Being able to compensate the distortion of ERS effect would make it possible to build high resolution, high frame rate and still inexpensive video cameras.

To some extent the effect of moving camera can be compensated by post-processing of the video, estimating the movement of the camera by comparing consecutive frames and assuming that the accelerations (changes in camera movement/rotation speed) during a single frame were low. This seem to be implemented in [http://www.guthspot.se/video/deshaker.htm#rolling%20shutter%20setting Deshaker for VirtualDub]

The quality of correction could be higher if the movement of the camera was tracked with higher temporal resolution. There are other applications that require high resolution (and precision) imagery that could benefit from such system. This can be tested by converting a camera (or one of the several sensors of the same camera - see [[10353#10359|sensor multiplexer board]]) into an "optical mouse". Regular optical mice have tiny cameras (with usually just 16x16 or 32x32 pixel resolution) running at a high frame rate and calculating correlation between images. Similar could be done with the Elphel reconfigurable hardware - run a small window on a regular sensor board and port/implement correlation code in the FPGA.

Supplementing recorded images/video from the main sensor with precise orientation/position of the camera during each line exposure will allow correction of the ERS distortion during post-processing.

=== Demosaic Algorithms in FPGA ===
What is it and why it is needed in [http://en.wikipedia.org/wiki/Demosaicing Demosaicing] in Wikipedia.
In Elphel cameras Bayer-encoded pixels are processed just in front of the compressor (JPEG/MJPEG, Ogg Theora). These compressors use 16x16 blocks of pixels converted to YCbCr (intensity and 2 color components), currently we use 4:2:0 that means that color (chroma) components have twice less the spacial resolution (in each direction) than intensity (luma). So for each 4 input pixels compressor needs 4 Y (luma) pixels and one of each Cb and Cr (chroma) ones.

Our first cameras used very simple algorithm to calculate YCbCr from the Bayer pixels and for each pixel it needed just 3x3 block of neighbors. And as these neighbors were needed for the outer pixels in the 16x16 blocks, FPGA had to read larger (18x18) overlapping blocks from the external memory (internal FPGA memory is much smaller and can not hold the whole image). In the later FPGA code 20x20 blocks are read in (to make possible implementation of the fancier demosaic algorithms), but that was not done - the outer pixels are discarded and still only 3x3 used.

There are several algorithms that provide good results with less artifacts (see Wikipedia article) and these detailed descriptions:
* [http://scien.stanford.edu/class/psych221/projects/99/tingchen/algodep/vargra.html Variable Number of Gradients]
* [http://web.cecs.pdx.edu/~cklin/demosaic/ Pixel Grouping]

So just implement one in the FPGA code of the camera? Or adapt those ideas to use use 4:2:0 encoding and convert to direct Bayer->YCbCr conversion (not Bayer->RGB->YCbCr)
=== Stereo Vision for Robots ===
For the 353 series of cameras we had developed a [[10353#10359|multiplexer board]] that can accommodate several sensor boards. As these boards are connected to the same FPGA it is rather easy to achieve a complete synchronization of the two sensors - i.e. just by skipping clock pulses to the sensors until their output frame sync pulses will match. When a pair of sensors is mechanically aligned some stereo processing can be performed on a line-by-line basis, storing the intermediate results in attached SDRAM chip and then improving the results by combining 1-d correlation data from multiple lines.

=== Visual Processing with the FPGA ===
Many machine vision algorithms can't be run on large image sizes in real time because of simple parallel-izable initial preprocessing passes; some of these could probably be conjoined with the color balancing and compression passes the elphel FPGA takes. A lot of research and products are built on top of the OpenCV libraries (initially released by intel and optimized for their processors but now under a BSD license and developed for several platforms; [http://www.youtube.com/watch?v=SNW0_wx_eHU simple example]). If parts of the library were reoptimized to take advantage of the FPGA, existing higher level code should get better performance with minimal tweaking.

Real-time artistic visual filters should also be possible at higher resolutions and/or frame rates than currently possible. A classic example is the waking life film which implemented ([http://www.hollywoodjesus.com/movie/waking_life/11.jpg example frame]) or the newer a scanner darkly ([http://www.apple.com/trailers/warner_independent_pictures/ascannerdarkly/large.html trailer]). These both had a lot of rotoscoping by hand, but good effects can be had automatically: gaussian blur, edge detect, blur more, and reduce color pallet to ~4-5.

Resources:
* [http://www.cs.colostate.edu/cameron/applications.html Cameron project]: Old attempt at doing visual filters on an FPGA (2002?)
* [http://www.intel.com/technology/computing/opencv/index.htm OpenCV] project at Intel: [http://en.wikipedia.org/wiki/OpenCV wikipedia article], [http://opencvlibrary.sourceforge.net/ wiki]

User:Bnewbold

2007-06-27T01:00:07Z

Bnewbold: /* SCINI Project */

= About bnewbold =
Hi! I'm an undergraduate physics major who also loves robotics and machine vision. I'm working on project for work using an Elphel camera right now, but i'm also very interested in implementing visual recognition and artistic filters on the Spartan FPGA for real time video art fun! My website is at http://bryannewbold.com

= SCINI Project =
SCINI is a submersible robot i'm helping develop that will probably use an Elphel video camera for control video and static imaging. See [[Research#SCINI:_Submersible_Capable_of_under_Ice_Navigation_and_Imaging]] and http://scini.mlml.calstate.edu for more information about the project. This space will track some of my experiences with the camera, questions i've got, etc. Feel free to copy anything from this page into other sections of the wiki if you find it useful and I haven't copied it over yet.

=== 12v Conversion ===
We want to run the camera off of 12v DC eventually, so we're trying to convert our POE 10353 revision C board to 12v. We were able to get a 12v to 3.3v DC/DC converter of the same footprint/model from [http://www.pduke.com/ P-Duke] because nobody else sold the part in small quantities; they mailed us a few but it didn't sound like they did small orders regularly. The part number was FKC08-12S33.

=== Linux Setup ===
Running the liveCD was a great way to see if everything was working, but the performance was unacceptable, so we installed ubuntu on a laptop and are using that. Both ubuntu and knoppix are debian based, and once I installed enough development packages on ubuntu I was able to install the elphel programs.

The netcamconfig script needed some small tweaks (remove knoppix references).

Performance of of the cgi camera control through firefox is still pretty slow, even when tested from a higher performance desktop: the bottle neck is probably the Elphel webserver sending full image photos in between accepting configuration commands. I'm thinking of writing a quick python/wxWidgets utility to change exposure and frame size settings faster; is anybody working on something similar? It could either drive ccam.cgi, the forthcoming PHP interface, or I could hack ccam.cgi to accept a non-HTTP TCP connection (''much'' smaller packets hopefully. maybe use Twisted on the control side?), or reimplement ccam.cgi in python (probably a bad idea to keep an extra python namespace running on the camera).

=== Streaming ===
On our ubuntu laptop I tried recompiling mplayer to accept huge RTP packets using the given patch from elphel@sourceforge as well as the ubuntu standard patch set, but it isn't all quite working yet. It took a long time to get all the required development tools installed on ubuntu (damn binary based distributions!) and I think I messed up the flag for sdp somewhere along the line. I was able to capture a few clips with mencoder.

Plain old VLC has been working great though, it can easily handle half resolution video at 12-15fps. With full resolution it throws some errors about long packets and the video comes out looking like this (note: it is video and the contents of the stripes change):
<center>[[Image:Vlcsnap-193174.png|400px]]</center>

I don't know if that's because of:
* packet data is being lost
* vlc can't handle the pixel count
* elphel is throwing up
* something else

=== Lens ===
We're looking for a small, wide angle lens that will mount onto the sensor board and still fit inside our forward looking dome. Water acts as a magnifier, so we need at least 90 degrees FOV.

User:Bnewbold

2007-06-26T21:58:26Z

Bnewbold: /* Streaming */

File:Vlcsnap-193174.png

2007-06-26T21:55:57Z

Bnewbold:

User:Bnewbold

2007-06-26T21:50:18Z

Bnewbold:

= About bnewbold =
Hi! I'm an undergraduate physics major who also loves robotics and machine vision. I'm working on project for work using an Elphel camera right now, but i'm also very interested in implementing visual recognition and artistic filters on the Spartan FPGA for real time video art fun! My website is at http://bryannewbold.com

= SCINI Project =
SCINI is a submersible robot i'm helping develop that will probably use an Elphel video camera for control video and static imaging. See [[Research#SCINI:_Submersible_Capable_of_under_Ice_Navigation_and_Imaging]] and http://scini.mlml.calstate.edu for more information about the project. This space will track some of my experiences with the camera, questions i've got, etc. Feel free to copy anything from this page into other sections of the wiki if you find it useful and I haven't copied it over yet.

=== 12v Conversion ===
We want to run the camera off of 12v DC eventually, so we're trying to convert our POE 10353 revision C board to 12v. We were able to get a 12v to 3.3v DC/DC converter of the same footprint/model from [http://www.pduke.com/ P-Duke] because nobody else sold the part in small quantities; they mailed us a few but it didn't sound like they did small orders regularly. The part number was FKC08-12S33.

=== Linux Setup ===
Running the liveCD was a great way to see if everything was working, but the performance was unacceptable, so we installed ubuntu on a laptop and are using that. Both ubuntu and knoppix are debian based, and once I installed enough development packages on ubuntu I was able to install the elphel programs.

The netcamconfig script needed some small tweaks (remove knoppix references).

Performance of of the cgi camera control through firefox is still pretty slow, even when tested from a higher performance desktop: the bottle neck is probably the Elphel webserver sending full image photos in between accepting configuration commands. I'm thinking of writing a quick python/wxWidgets utility to change exposure and frame size settings faster; is anybody working on something similar? It could either drive ccam.cgi, the forthcoming PHP interface, or I could hack ccam.cgi to accept a non-HTTP TCP connection (''much'' smaller packets hopefully. maybe use Twisted on the control side?), or reimplement ccam.cgi in python (probably a bad idea to keep an extra python namespace running on the camera).

=== Streaming ===
On our ubuntu laptop I tried recompiling mplayer to accept huge RTP packets using the given patch from elphel@sourceforge as well as the ubuntu standard patch set, but it isn't all quite working yet. It took a long time to get all the required development tools installed on ubuntu (damn binary based distributions!) and I think I messed up the flag for sdp somewhere along the line. I was able to capture a few clips with mencoder.

Plain old VLC has been working great though, it can easily handle half resolution video at 12-15fps. With full resolution it throws some errors about long packets and the video comes out looking like this (note: it is video and the contents of the stripes change):

I don't know if that's because of:
* packet data is being lost
* vlc can't handle the pixel count
* elphel is throwing up
* something else

UserProjects

2007-06-26T20:27:30Z

Bnewbold: /* SCINI: Submersible Capable of under Ice Navigation and Imaging */

== Research Projects ==
There are several research and educational projects that use Elphel cameras. Some are already active, others just starting or only in planning stage. We believe that scientific research is the application area where the open nature of our products is the most beneficial. The shortage of the powerful and at the same time open and flexible hardware was that "itch" that led me to start Elphel.

This page is started to provide resources to those who already uses or plans to use Elphel cameras. This is wiki, so please feel free to add you project description/link below.--[[User:Andrey.filippov|Andrey.filippov]] 14:59, 29 April 2007 (CDT)

==== SCINI: Submersible Capable of under Ice Navigation and Imaging ====
A group at the [http://www.mlml.calstate.edu Moss Landing Marine Labs] in Moss Landing, CA (USA) is working on a new submersible ROV (remotely operated vehicle) which will be first deployed in Antarctica in the fall of 2007; it is a three year development program ending in 2009.

They are planning on using an Elphel 353 video camera for the main navigation camera, and possibly adding on a second high resolution still camera sensor board for image collection. The ability to run video over ethernet with the rest of the communications means the 1000 foot (330 meter) tether does not need dedicated conductors for video transmission, which saves a lot of hassle! The configurability of exposure, color balance, frame rate, etc, through open standards and software means less reverse engineering and easier integration with existing robotics control software.

<center>[[Image:Camera_module_sm.jpg]]

Prototype Waterproof Camera Bottle</center>

For more information about the project including a blog see http://scini.mlml.calstate.edu/

[[User:Bnewbold]] is the main contact on this wiki for the SCINI Project; see his talk page for Elphel-specific questions and software related to the SCINI Project.

File:Camera module sm.jpg

2007-06-26T20:23:13Z

Bnewbold:

UserProjects

2007-06-26T20:21:02Z

Bnewbold: /* Research Projects */

== Research Projects ==
There are several research and educational projects that use Elphel cameras. Some are already active, others just starting or only in planning stage. We believe that scientific research is the application area where the open nature of our products is the most beneficial. The shortage of the powerful and at the same time open and flexible hardware was that "itch" that led me to start Elphel.

This page is started to provide resources to those who already uses or plans to use Elphel cameras. This is wiki, so please feel free to add you project description/link below.--[[User:Andrey.filippov|Andrey.filippov]] 14:59, 29 April 2007 (CDT)

==== SCINI: Submersible Capable of under Ice Navigation and Imaging ====
A group at the [http://www.mlml.calstate.edu Moss Landing Marine Labs] in Moss Landing, CA (USA) is working on a new submersible ROV (remotely operated vehicle) which will be first deployed in Antarctica in the fall of 2007; it is a three year development program ending in 2009.

They are planning on using an Elphel 353 video camera for the main navigation camera, and possibly adding on a second high resolution still camera sensor board for image collection. The ability to run video over ethernet with the rest of the communications means the 1000 foot (330 meter) tether does not need dedicated conductors for video transmission, which saves a lot of hassle! The configurability of exposure, color balance, frame rate, etc, through open standards and software means less reverse engineering and easier integration with existing robotics control software.

<center>[[Image:camera_module.jpg]]
Prototype Waterproof Camera Bottle</center>

Roadmap

2007-05-31T21:17:02Z

Bnewbold: /* Camera hardware */

== Background ==
Elphel was started in 2001 (Magna, UT USA) when I quit my job inspired by the possibilities that Free Software gives to developers (it was not so obvious for my at the time employer). All the projects were covered in [http://www.linuxdevices.com LinuxDevices] (complete list on the articles is available [http://www.elphel.com/articles/index.html here]). For several years Elphel was a one-man company, in January 2004 I wrote an article [http://www.computerra.ru/hitech/tech/31862/ Taming of the Iron Penguin (Russian)] in the largest Russain computer-related magazine [http://www.computerra.ru Computerra] and announced there a competition among the software developers for the best video streamer to run in the camera. That was a good idea and after the competition itself was over most of the developers remained in the Elphel team. At first - as volunteers, later - as full/part time employees.

Not all of these developers live in Russia - two, including the winner of the competition are from Kiev, Ukraine. But still all of them know Russian much better than English and so most of our technical discussions were on our private Russian-language forum. So far I failed to move these discussions to the broader audience but believe that Wiki technology can help. Here we will mantain most of the site in English but still have some pages/discussions in Russian, translating documents as we go. Or when somebody else needs it and is not satisfied with [http://babelfish.altavista.com Babelfish] automatic translation. We will try to keep English pages current - anyway even in Elphel not everybody knows Russian.

Please excuse not-so-good English of our developers and feel free to fix the errors if you see them.

--[[User:Andrey.filippov|Andrey.filippov]] 15:34, 22 September 2005 (CST)
== Update on 353/363 cameras ==
There is preliminary documentation for [[353|the new model 353/353 cameras hardware here]]

353/363 cameras use software based on Linux kernel 2.6 (currently - 2.6.15). There will be more changes in software as we used different web server (lighttpd) and included PHP (as fastCGI application).

More information on software will follow, you may already find code here:
* [http://sourceforge.net/project/showfiles.php?group_id=105686&package_id=222883 elphel353 software package @ Sourceforge]
* [http://elphel.cvs.sourceforge.net/elphel/elphel353/elphel353 CVS @ Sourceforge]

== Software Architecture of Elphel 3x3 cameras ==
Software in the Elphel cameras started from [http://developer.axis.com/ Axis Developer Boards Software] and was amended for the camera specific functions. It was modified to work with newer hardware (models 303-313/323-333), support more features and now seems to be a good time to make a major redesign instead of applying incremental changes.

Some discussion already started in Russian here - [[Nc3x3]]

Related to the architecture are the [[#Camera Interface]] and the [[#Client Software]]

Elphel will continue developing a web browser based user interface with [http://en.wikipedia.org/wiki/AJAX AJAX] technology. That will require developing/modifying player plugins controllable from [http://en.wikipedia.org/wiki/Javascript JavaScript] and implementing specific features needed for video surveillance applications - multiple camera views on the same page, digital PTZ (inside the hi-res incoming stream) and temporal decimation (reducing frame rate) that uses as little CPU resources as possible.

The web-based user interface can be especially useful for open hardware as it reduces the entrance threshold for the developer who would like to customize the cameras functionality - regular web development tools are sufficient for the job.

=== Camera Interface ===

The camera now has two alternative APIs:

==== ccam.cgi ====

The original interface that supports most camera features - [[ccam.cgi]]

and

==== API compatible with Axis cameras ====

This ([[AxisAPI]]) makes Elphel cameras work with some third-party software

==== JavaScript library ====
We will create a set of javascript routines to control the cameras, which can be used in a different AJAX applications.
See [[JavaScript API]]

=== Camera Software ===
==== File systems ====
[[333_File_System]]

=== Client Software ===
==== [[Talk:Recorder|Recorder]] ====
We will start a new recorder for MJPEG RTP stream.
It will be small standalone program, which will take a multicast IP address and port, receive a stream and record it to files or to stdout.
The most important thing with the recorder - to avoid frame drops (maybe something can be done with network buffering) and control them using timestamps.
And the recorder must provide managable files (under 2GB), but with zero drops between them.

==== Plugins for browsers ====
[[GenReS|GenReS plugin]]

==== [[MPlayer]] ====
We have MPlayer patched for use with our cameras. Patches for source code are accessible on Source Forge but the compiled package is only for Debian on i386 architecture. We plan to make compiled packages for PowerPC architecture and also for Slackware.

==== [[HTML Video Surveillance]] ====
[http://sourceforge.net/project/showfiles.php?group_id=105686&package_id=138717&release_id=358392 Multiple camera view HTML page] is based on [[GenReS |GenReS plugin]] for [http://www.mozilla.org/ Mozilla/FireFox].
Currently working: scrolling by picture dragging (digital PTZ), camera selection, zoom switch, automatic detection of stream stop by timeout.
List of camera adresses is now editable manually. It will be automaticaly generated in the [[Live CD]].
The page will run recording software by user request. Video will be saved to a fixed directory and split into separate files by tunable number of frames.
Main parameters of video capture will be changeable from the page.
The page can later be used in the [[#Video Server]].

==== [[Live CD]] ====
The Elphel live Linux CD contains software for camera users. We also will make a live DVD for developers.
The live CD is based on [http://knopper.net/knoppix/ Knoppix].
<ul>The software which should be included in future releases of the Live CD
<li>[[HTML Video Surveillance]]
<li>[http://lives.sourceforge.net LiVES] video editor
<li>client software packages with simple installation for different distributions of GNU/Linux
</ul>
Currently we only have CDs for the i386 architecture.

We have plans to make a Live CD for PowerPC too.

We should move to DVD distribution as most of the disks are already provided with the hardware, not downloaded.

The idea of keeping as full a Knoppix as possible was to introduce GNU/Linux to the camera users who had never had this experience before. But these users will get a DVD in a box, the downloadable CD version can have more standard packages removed and replaced with camera-specific software.

One of the major additions will be a preinstalled camera development environment (possibly based on [http://www.eclipse.org Eclipse]) to simplify the modification of the camera code. Again - don't forget that many of those future developers now only use Visual Studio (or whatever it is called?) and GNU/Linux can be somewhat alien to them. This environment allows them to start playing with their code without prior knowledge of the GNU/Linux software development process.

It can be useful for hardware/fpga developers too - to be able to write some code to support the hardware features without spending too much time on mastering the software development process.

see [[LiveCD Release Notes]] for schedule

===== LiveCD remastering =====

[[Livecd-make]] - Script for the original Knoppix CD or DVD remastering (author - Khlutchin Sergey)

[[Live-CD remastering]] - Scripts for Knoppix CD remastering only (author - Ilya Shpankov)

=== Video Server ===
PC-based video server that will archive incoming Ogg Theora incoming streams from several cameras and transcode them on the fly to lower resolution (binary decimation, windowing) and frame rate (i.e. using only key frames) presenting multiple streams (real time and recorded) to the operator. The external interface of the server might be one of the industry standard ones and be compatible with 3rd party legacy software.

== Camera hardware ==
==== [[353]]/363 series ====
'''Basic:''' [[10353]] (processor board w/ CPU and FPGA), [[10338]] (sensor board for Micron MT9P031/MT9P001 5MPix)

'''For Kodak CCD:''' [[10342]] (CCD Interface Board), [[10347]]
'''Add ons:''' [[10357]] (solid state memory), [[10359]] (multi-sensor board/multifunction multiplexer)
==== Other Hardware ====
[[RTC]] (real time clock), [[10331]], [[10332]], [[10334]] (USB lens control)

== Active Projects ==
=== Synchronization of the Cameras ===

Sometimes you need to acquire images triggered by an extarnal event or several cameras need to be synchronized with each other. [[Camera Synchronization]] is all about this.

=== Photo-finish ===
Photo-finish device made using Elphel model 333 camera with additional FPGA code and software - [[Photo-finish]]

=== Zeroconf for Elphel cameras ===
[[zeroconf for Elphel cameras]]
=== Elphel cameras and Zoneminder ===
We plan to make the model 333 camera work with [http://www.zoneminder.com Zoneminder]
=== USB host interface ===

daughter board with USB and DC-DC power for lens control board [[10334]]

Proposal for a usb audio solution : [[PCM2903]]

=== Motorized lens control ===
I'll try to retrieve what was written before on the motorized lens control. In short - C/CS mount is rather old and does not work well for interchangeable motorized lenses. We are trying to build an adapter from C/CS-mount to a bayonet type connector. And place a tiny 5mm wide PCB ring in that adapter. This [[10331]] PCB has a reprogrammable microcontroller and uses just 2 connections to the camera for power and data signals combined. It provides all the necessary connections for most types of motorized lenses.

lens control board [[10331]]
DC-DC power board for motorized lens control board [[10332]]
lens control board In System Programmer [[lbcontrol]]

=== Outdoor enclosure ===

'''Step Zero'''

Determine working setup
-Does the system need a control board
-CCD board needs a longer cable for minimal package when stacking lens on top of board [http://www.maartenmenheere.nl/blog/images/014-001-0.jpg Camera casing]

'''Step one'''
Test setup. Assemble all components in a setup that can record video

Components in test setup
- Lens (Computar H3Z4512CS varifocal lens? using power)
- Elphel USB setup. Is it possible to directly plug in a usb drive. Where does the power come from?
- Battery
- Usb cable or network calbe
- Usb exteral harddrive or flashdrive
- ON/off switch

Objective: Does it work, at all?
Secondary: Battery life? Video quality?

'''Step two'''

Wooden box. Test setup 1 integrated in outside video testing setup.

Components added in test 2
- Hardboard casing

Objective: Optimize recording setup of video for ease of use
Secondary: optimal settings? correct lens?

'''Step three'''

Building of waterproof casing
- Amphenol plugs
- Camera window
- Casing camera (fibre reinforced composite)
- Casing base station (battery + storage) (fibre reinforced composite)

[http://www.maartenmenheere.nl/blog/images/outdoorvideosystem.jpg Schematic]
[http://www.maartenmenheere.nl/blog/images/014-001-0.jpg Camera casing]
[http://www.maartenmenheere.nl/blog/images/camera.jpg Camera casing]
[http://www.maartenmenheere.nl/blog/images/Radjump_elphel.jpg Outdoor enclosure]
[http://www.maartenmenheere.nl/blog/images/CamAssy-Sheet1.pdf Camera casing 2]

=== Current enclosure design ===

We are switching to extruded aluminum tube (actually original 303/313 also was design for a standard aluminum profile). Model 333 RJ-45 connector is designed to fit into RJField shell [http://www.rjfield.com/ethernet_connectors_rjf_en.htm].

[[Mechanical]]

=== Removal of distortion ===

Distortions from the non-instantaneous exposure of the frame can be done in [http://lives.sf.net LiVES]. But first some other infrastructure must be in place:

- Camera must start a [http://www.piksel.no/pwiki/VideoJack videojack] server on the host machine, with the correct fps, width, height and frame palette

- Camera must activate the videojack receiver in LiVES and connect it the correct videojack server

- Camera can start to send unpackaged compressed frames to videojack server, along with an array of floats

- Floats will be in pairs for each horizontal band: the vertical compression/expansion (1.0 means no compression) and the horizontal shift (+-shift/width)

- LiVES will pull these frames and the float array from the videojack server

- LiVES will decode each frame and pass it along with the float array to a [http://www.xs4all.nl/~salsaman/lives/docs/weedspec.txt Weed] effect which will apply the compress/expand/shift

- LiVES will receive the altered frame and save it to a stream
==== Post-processing - non realtime ====
CMOS sensor-specific distortions caused by the [[Electronic Rolling Shutter ]] do not need to be removed realtime - the PC power may be not enough to do so. It is perfectly OK to be able to just post-process the video recorded from the camera.

--[[User:Andrey.filippov|Andrey.filippov]] 02:39, 9 December 2005 (CST)

== Known problems ==

[[Known problems]]

== Diagnostic and repair ==

[[Diagnostic and repair]] page in the Cameras Manual

=== Changing or restoring the MAC address ===

The factory default MAC address of Elphel products is the same as the serial number which begin with 000E64. If you would like to use a MAC address different from the serial number you can edit the /etc/conf.d/mac file.

If the serial number has not been configured is possible to set it (but only once!). To set the serial number use the bootblocktool command on the camera:

<nowiki>bootblocktool -a SERNO 000E64xxxxxx</nowiki>

Exchange “xxxxxx” with the serial number of the board.

!As you can do it only once be careful! If you write the MAC address in a wrong format (ie with : separations) the camera will not boot any more.

You may do it with the special HTML [http://www.elphel.com/3fhlo/bootblock/bootblock333.html page] but those serial numbers are supposed to be factory programmed.--[[User:Andrey.filippov|Andrey.filippov]] 05:36, 30 December 2005 (CST)

Roadmap

2007-05-31T21:14:46Z

Bnewbold: /* Camera hardware */

== Background ==
Elphel was started in 2001 (Magna, UT USA) when I quit my job inspired by the possibilities that Free Software gives to developers (it was not so obvious for my at the time employer). All the projects were covered in [http://www.linuxdevices.com LinuxDevices] (complete list on the articles is available [http://www.elphel.com/articles/index.html here]). For several years Elphel was a one-man company, in January 2004 I wrote an article [http://www.computerra.ru/hitech/tech/31862/ Taming of the Iron Penguin (Russian)] in the largest Russain computer-related magazine [http://www.computerra.ru Computerra] and announced there a competition among the software developers for the best video streamer to run in the camera. That was a good idea and after the competition itself was over most of the developers remained in the Elphel team. At first - as volunteers, later - as full/part time employees.

Not all of these developers live in Russia - two, including the winner of the competition are from Kiev, Ukraine. But still all of them know Russian much better than English and so most of our technical discussions were on our private Russian-language forum. So far I failed to move these discussions to the broader audience but believe that Wiki technology can help. Here we will mantain most of the site in English but still have some pages/discussions in Russian, translating documents as we go. Or when somebody else needs it and is not satisfied with [http://babelfish.altavista.com Babelfish] automatic translation. We will try to keep English pages current - anyway even in Elphel not everybody knows Russian.

Please excuse not-so-good English of our developers and feel free to fix the errors if you see them.

--[[User:Andrey.filippov|Andrey.filippov]] 15:34, 22 September 2005 (CST)
== Update on 353/363 cameras ==
There is preliminary documentation for [[353|the new model 353/353 cameras hardware here]]

353/363 cameras use software based on Linux kernel 2.6 (currently - 2.6.15). There will be more changes in software as we used different web server (lighttpd) and included PHP (as fastCGI application).

More information on software will follow, you may already find code here:
* [http://sourceforge.net/project/showfiles.php?group_id=105686&package_id=222883 elphel353 software package @ Sourceforge]
* [http://elphel.cvs.sourceforge.net/elphel/elphel353/elphel353 CVS @ Sourceforge]

== Software Architecture of Elphel 3x3 cameras ==
Software in the Elphel cameras started from [http://developer.axis.com/ Axis Developer Boards Software] and was amended for the camera specific functions. It was modified to work with newer hardware (models 303-313/323-333), support more features and now seems to be a good time to make a major redesign instead of applying incremental changes.

Some discussion already started in Russian here - [[Nc3x3]]

Related to the architecture are the [[#Camera Interface]] and the [[#Client Software]]

Elphel will continue developing a web browser based user interface with [http://en.wikipedia.org/wiki/AJAX AJAX] technology. That will require developing/modifying player plugins controllable from [http://en.wikipedia.org/wiki/Javascript JavaScript] and implementing specific features needed for video surveillance applications - multiple camera views on the same page, digital PTZ (inside the hi-res incoming stream) and temporal decimation (reducing frame rate) that uses as little CPU resources as possible.

The web-based user interface can be especially useful for open hardware as it reduces the entrance threshold for the developer who would like to customize the cameras functionality - regular web development tools are sufficient for the job.

=== Camera Interface ===

The camera now has two alternative APIs:

==== ccam.cgi ====

The original interface that supports most camera features - [[ccam.cgi]]

and

==== API compatible with Axis cameras ====

This ([[AxisAPI]]) makes Elphel cameras work with some third-party software

==== JavaScript library ====
We will create a set of javascript routines to control the cameras, which can be used in a different AJAX applications.
See [[JavaScript API]]

=== Camera Software ===
==== File systems ====
[[333_File_System]]

=== Client Software ===
==== [[Talk:Recorder|Recorder]] ====
We will start a new recorder for MJPEG RTP stream.
It will be small standalone program, which will take a multicast IP address and port, receive a stream and record it to files or to stdout.
The most important thing with the recorder - to avoid frame drops (maybe something can be done with network buffering) and control them using timestamps.
And the recorder must provide managable files (under 2GB), but with zero drops between them.

==== Plugins for browsers ====
[[GenReS|GenReS plugin]]

==== [[MPlayer]] ====
We have MPlayer patched for use with our cameras. Patches for source code are accessible on Source Forge but the compiled package is only for Debian on i386 architecture. We plan to make compiled packages for PowerPC architecture and also for Slackware.

==== [[HTML Video Surveillance]] ====
[http://sourceforge.net/project/showfiles.php?group_id=105686&package_id=138717&release_id=358392 Multiple camera view HTML page] is based on [[GenReS |GenReS plugin]] for [http://www.mozilla.org/ Mozilla/FireFox].
Currently working: scrolling by picture dragging (digital PTZ), camera selection, zoom switch, automatic detection of stream stop by timeout.
List of camera adresses is now editable manually. It will be automaticaly generated in the [[Live CD]].
The page will run recording software by user request. Video will be saved to a fixed directory and split into separate files by tunable number of frames.
Main parameters of video capture will be changeable from the page.
The page can later be used in the [[#Video Server]].

==== [[Live CD]] ====
The Elphel live Linux CD contains software for camera users. We also will make a live DVD for developers.
The live CD is based on [http://knopper.net/knoppix/ Knoppix].
<ul>The software which should be included in future releases of the Live CD
<li>[[HTML Video Surveillance]]
<li>[http://lives.sourceforge.net LiVES] video editor
<li>client software packages with simple installation for different distributions of GNU/Linux
</ul>
Currently we only have CDs for the i386 architecture.

We have plans to make a Live CD for PowerPC too.

We should move to DVD distribution as most of the disks are already provided with the hardware, not downloaded.

The idea of keeping as full a Knoppix as possible was to introduce GNU/Linux to the camera users who had never had this experience before. But these users will get a DVD in a box, the downloadable CD version can have more standard packages removed and replaced with camera-specific software.

One of the major additions will be a preinstalled camera development environment (possibly based on [http://www.eclipse.org Eclipse]) to simplify the modification of the camera code. Again - don't forget that many of those future developers now only use Visual Studio (or whatever it is called?) and GNU/Linux can be somewhat alien to them. This environment allows them to start playing with their code without prior knowledge of the GNU/Linux software development process.

It can be useful for hardware/fpga developers too - to be able to write some code to support the hardware features without spending too much time on mastering the software development process.

see [[LiveCD Release Notes]] for schedule

===== LiveCD remastering =====

[[Livecd-make]] - Script for the original Knoppix CD or DVD remastering (author - Khlutchin Sergey)

[[Live-CD remastering]] - Scripts for Knoppix CD remastering only (author - Ilya Shpankov)

=== Video Server ===
PC-based video server that will archive incoming Ogg Theora incoming streams from several cameras and transcode them on the fly to lower resolution (binary decimation, windowing) and frame rate (i.e. using only key frames) presenting multiple streams (real time and recorded) to the operator. The external interface of the server might be one of the industry standard ones and be compatible with 3rd party legacy software.

== Camera hardware ==
==== [[353]]/363 series ====
'''Basic:''' [[10353]] (processor board w/ CPU and FPGA)

'''For Kodak CCD:''' [[10342]] (CCD Interface Board), [[10347]]
'''Add ons:''' [[10357]] (solid state memory), [[10359]] (multi-sensor board/multifunction multiplexer)
==== Other Hardware ====
[[RTC]] (real time clock), [[10331]], [[10332]], [[10334]] (USB lens control)

== Active Projects ==
=== Synchronization of the Cameras ===

Sometimes you need to acquire images triggered by an extarnal event or several cameras need to be synchronized with each other. [[Camera Synchronization]] is all about this.

=== Photo-finish ===
Photo-finish device made using Elphel model 333 camera with additional FPGA code and software - [[Photo-finish]]

=== Zeroconf for Elphel cameras ===
[[zeroconf for Elphel cameras]]
=== Elphel cameras and Zoneminder ===
We plan to make the model 333 camera work with [http://www.zoneminder.com Zoneminder]
=== USB host interface ===

daughter board with USB and DC-DC power for lens control board [[10334]]

Proposal for a usb audio solution : [[PCM2903]]

=== Motorized lens control ===
I'll try to retrieve what was written before on the motorized lens control. In short - C/CS mount is rather old and does not work well for interchangeable motorized lenses. We are trying to build an adapter from C/CS-mount to a bayonet type connector. And place a tiny 5mm wide PCB ring in that adapter. This [[10331]] PCB has a reprogrammable microcontroller and uses just 2 connections to the camera for power and data signals combined. It provides all the necessary connections for most types of motorized lenses.

lens control board [[10331]]
DC-DC power board for motorized lens control board [[10332]]
lens control board In System Programmer [[lbcontrol]]

=== Outdoor enclosure ===

'''Step Zero'''

Determine working setup
-Does the system need a control board
-CCD board needs a longer cable for minimal package when stacking lens on top of board [http://www.maartenmenheere.nl/blog/images/014-001-0.jpg Camera casing]

'''Step one'''
Test setup. Assemble all components in a setup that can record video

Components in test setup
- Lens (Computar H3Z4512CS varifocal lens? using power)
- Elphel USB setup. Is it possible to directly plug in a usb drive. Where does the power come from?
- Battery
- Usb cable or network calbe
- Usb exteral harddrive or flashdrive
- ON/off switch

Objective: Does it work, at all?
Secondary: Battery life? Video quality?

'''Step two'''

Wooden box. Test setup 1 integrated in outside video testing setup.

Components added in test 2
- Hardboard casing

Objective: Optimize recording setup of video for ease of use
Secondary: optimal settings? correct lens?

'''Step three'''

Building of waterproof casing
- Amphenol plugs
- Camera window
- Casing camera (fibre reinforced composite)
- Casing base station (battery + storage) (fibre reinforced composite)

[http://www.maartenmenheere.nl/blog/images/outdoorvideosystem.jpg Schematic]
[http://www.maartenmenheere.nl/blog/images/014-001-0.jpg Camera casing]
[http://www.maartenmenheere.nl/blog/images/camera.jpg Camera casing]
[http://www.maartenmenheere.nl/blog/images/Radjump_elphel.jpg Outdoor enclosure]
[http://www.maartenmenheere.nl/blog/images/CamAssy-Sheet1.pdf Camera casing 2]

=== Current enclosure design ===

We are switching to extruded aluminum tube (actually original 303/313 also was design for a standard aluminum profile). Model 333 RJ-45 connector is designed to fit into RJField shell [http://www.rjfield.com/ethernet_connectors_rjf_en.htm].

[[Mechanical]]

=== Removal of distortion ===

Distortions from the non-instantaneous exposure of the frame can be done in [http://lives.sf.net LiVES]. But first some other infrastructure must be in place:

- Camera must start a [http://www.piksel.no/pwiki/VideoJack videojack] server on the host machine, with the correct fps, width, height and frame palette

- Camera must activate the videojack receiver in LiVES and connect it the correct videojack server

- Camera can start to send unpackaged compressed frames to videojack server, along with an array of floats

- Floats will be in pairs for each horizontal band: the vertical compression/expansion (1.0 means no compression) and the horizontal shift (+-shift/width)

- LiVES will pull these frames and the float array from the videojack server

- LiVES will decode each frame and pass it along with the float array to a [http://www.xs4all.nl/~salsaman/lives/docs/weedspec.txt Weed] effect which will apply the compress/expand/shift

- LiVES will receive the altered frame and save it to a stream
==== Post-processing - non realtime ====
CMOS sensor-specific distortions caused by the [[Electronic Rolling Shutter ]] do not need to be removed realtime - the PC power may be not enough to do so. It is perfectly OK to be able to just post-process the video recorded from the camera.

--[[User:Andrey.filippov|Andrey.filippov]] 02:39, 9 December 2005 (CST)

== Known problems ==

[[Known problems]]

== Diagnostic and repair ==

[[Diagnostic and repair]] page in the Cameras Manual

=== Changing or restoring the MAC address ===

The factory default MAC address of Elphel products is the same as the serial number which begin with 000E64. If you would like to use a MAC address different from the serial number you can edit the /etc/conf.d/mac file.

If the serial number has not been configured is possible to set it (but only once!). To set the serial number use the bootblocktool command on the camera:

<nowiki>bootblocktool -a SERNO 000E64xxxxxx</nowiki>

Exchange “xxxxxx” with the serial number of the board.

!As you can do it only once be careful! If you write the MAC address in a wrong format (ie with : separations) the camera will not boot any more.

You may do it with the special HTML [http://www.elphel.com/3fhlo/bootblock/bootblock333.html page] but those serial numbers are supposed to be factory programmed.--[[User:Andrey.filippov|Andrey.filippov]] 05:36, 30 December 2005 (CST)

10357

2007-05-31T20:59:52Z

Bnewbold: /* 10357 */ syntax

==10357==
[[Image:10357_top_tn.jpeg|frame|[[Media:10357_top.jpeg|10357 board, top view]]]]
[[Image:10357_bottom_tn.jpeg|frame|[[Media:10357_bottom.jpeg|10357 board, bottom view]]]]
[[Image:10357bd.png|frame|[[Media:10357sch.pdf|10357 Circuit Diagram]]]]
10357 board is designed to provide a solid-state storage medium for [[353|353/363-series cameras]]. It multiplexes the IDE bus (available from the [[10353|10353 board]]) between 4 master/slave pairs of CF cards configured to operate in IDE mode. With 8 of 16GB CF cards installed the 10357 board can provide 128GB of storage

On-board FPGA can be later configured to write data to several CF cards in parallel increasing the overall data rate.

In addition to the main function of CF storage 10357 can be also used for prototyping purposes. It has FPGA that is connected to auxiliary 30-pin connector and can be programmed by and can communicate with the software running on the main board. It has 4 groups of pairs of 50-pin headers, most pins are connected to individual FPGA pads and so can be programmed for a variety of I/O functions.

Talk:Change API legacy

2007-05-31T02:39:59Z

Bnewbold:

This doesn't seem to work, at least on the 353 series... 404 error.
Does the 353 even do the AxisAPI?--[[User:Bnewbold|Bnewbold]] 21:39, 30 May 2007 (CDT)