Difference between revisions of "Roadmap"

From ElphelWiki
Jump to: navigation, search
m (→‎Background: small typo fixed)
 
(99 intermediate revisions by 24 users not shown)
Line 1: Line 1:
 
== Background ==
 
== Background ==
−
I (Andrey Filippov) quit my job and started Elphel, Inc. in 2001 (Magna, UT USA), all the projects were covered in [http://www.linuxdevices.com LinuxDevices] (complete list on the articles is available [http://www.elphel.com/articles/index.html here]). For several years Elphel was a one-man company, in January 2004 I wrote an article [http://www.computerra.ru/hitech/tech/31862/ Taming of the Iron Penguin (Russian)] in the largest Russain computer-related magazine [http://www.computerra.ru Computerra] and announced there a competition among the software developers for the best video streamer to run in the camera. That was a good idea and after the competition itself was over most of the developers remained in the Elphel team. At first - as volunteers, later - as full/part time employees.
+
Elphel was started in 2001 (Magna, UT USA) when I quit my job inspired by the possibilities that Free Software gives to developers (it was not so obvious for my at the time employer). All the projects were covered in [http://www.linuxdevices.com LinuxDevices] (complete list on the articles is available [http://legacy.elphel.com/articles/index.html here]). For several years Elphel was a one-man company, in January 2004 I wrote an article [http://www.computerra.ru/hitech/tech/31862/ Taming of the Iron Penguin (Russian)] in the largest Russain computer-related magazine [http://www.computerra.ru Computerra] and announced there a competition among the software developers for the best video streamer to run in the camera. That was a good idea and after the competition itself was over most of the developers remained in the Elphel team. At first - as volunteers, later - as full/part time employees.
  
 
Not all of these developers live in Russia - two, including the winner of the competition are from Kiev, Ukraine. But still all of them know Russian much better than English and so most of our technical discussions were on our private Russian-language forum. So far I failed to move these discussions to the broader audience but believe that Wiki technology can help. Here we will mantain most of the site in English but still have some pages/discussions in Russian, translating documents as we go. Or when somebody else needs it and is not satisfied with [http://babelfish.altavista.com Babelfish] automatic translation. We will try to keep English pages current - anyway even in Elphel not everybody knows Russian.
 
Not all of these developers live in Russia - two, including the winner of the competition are from Kiev, Ukraine. But still all of them know Russian much better than English and so most of our technical discussions were on our private Russian-language forum. So far I failed to move these discussions to the broader audience but believe that Wiki technology can help. Here we will mantain most of the site in English but still have some pages/discussions in Russian, translating documents as we go. Or when somebody else needs it and is not satisfied with [http://babelfish.altavista.com Babelfish] automatic translation. We will try to keep English pages current - anyway even in Elphel not everybody knows Russian.
  
 
Please excuse not-so-good English of our developers and feel free to fix the errors if you see them.
 
Please excuse not-so-good English of our developers and feel free to fix the errors if you see them.
 +
 +
--[[User:Andrey.filippov|Andrey.filippov]] 15:34, 22 September 2005 (CST)
 +
== Update on 353/363 cameras ==
 +
There is preliminary documentation for [[353|the new model 353/353 cameras hardware here]]
 +
 +
353/363 cameras use software based on Linux kernel 2.6 (currently - 2.6.19). There will be more changes in software as we used different web server (lighttpd) and included PHP (as fastCGI application).
 +
 +
More information on software will follow, you may already find code here:
 +
* [http://sourceforge.net/project/showfiles.php?group_id=105686&package_id=222883 elphel353 software package @ Sourceforge]
 +
* [http://elphel.cvs.sourceforge.net/elphel/elphel353/elphel353 CVS @ Sourceforge]
  
 
== Software Architecture of Elphel 3x3 cameras ==
 
== Software Architecture of Elphel 3x3 cameras ==
−
Software in the Elphel cameras started from [http://developer.axis.com/ Axis Developer Boards Software] and was amended for the camera specific functions. It was modified to work with newer hardware (models 303-313/323-333), support more features and now it seems to be a good time to make a major redesign instead of applying incremental changes.
+
Software in the Elphel cameras started from [http://developer.axis.com/ Axis Developer Boards Software] and was amended for the camera specific functions. It was modified to work with newer hardware (models 303-313/323-333), support more features and now seems to be a good time to make a major redesign instead of applying incremental changes.
  
 
Some discussion already started in Russian here - [[Nc3x3]]
 
Some discussion already started in Russian here - [[Nc3x3]]
Line 13: Line 23:
 
Related to the architecture are the [[#Camera Interface]] and the [[#Client Software]]
 
Related to the architecture are the [[#Camera Interface]] and the [[#Client Software]]
  
−
Elphel will continue developing web browser based user interface with [http://en.wikipedia.org/wiki/AJAX AJAX] technique. That will require to develop/modify player plugins controllable from [http://en.wikipedia.org/wiki/Javascript JavaScript] and implementing specific features needed for video surveillance applications - multiple camera views on the same page, digital PTZ (inside the hi-res incoming stream) and temporal decimation (reducing frame rate) that uses as low CPU resources as possible.
+
Elphel will continue developing a web browser based user interface with [http://en.wikipedia.org/wiki/AJAX AJAX] technology. That will require developing/modifying player plugins controllable from [http://en.wikipedia.org/wiki/Javascript JavaScript] and implementing specific features needed for video surveillance applications - multiple camera views on the same page, digital PTZ (inside the hi-res incoming stream) and temporal decimation (reducing frame rate) that uses as little CPU resources as possible.
  
−
Web-based user interface can be especially useful for the open hardware as it reduces the entrance threshold for the developer who would like to customize the cameras functionality - regular web development tools are sufficient for the job.
+
The web-based user interface can be especially useful for open hardware as it reduces the entrance threshold for the developer who would like to customize the cameras functionality - regular web development tools are sufficient for the job.
  
 
=== Camera Interface ===
 
=== Camera Interface ===
  
−
Camera now has two alternative APIs:
+
The camera now has two alternative APIs:
  
 
==== ccam.cgi ====
 
==== ccam.cgi ====
  
−
Original interface that supports most camera features - [[ccam.cgi]]
+
The original interface that supports most camera features - [[ccam.cgi]]
  
 
and
 
and
  
−
==== API compatible with Axis cameras ====
+
==== PHP API ====
 +
 
 +
=====[[PHP_in_Elphel_cameras| Using PHP in Elphel camera]]=====
 +
 
 +
=====[[Elphel PHP constants]]=====
 +
 
 +
=====[[PHP Examples]]=====
  
−
This ([[AxisAPI]]) makes Elphel cameras work with some third-party software
+
=====[[Binning]]=====
  
 
==== JavaScript library ====
 
==== JavaScript library ====
−
We will create a set of javascript routines to control cameras, which can be used in a different AJAX applications.
+
We will create a set of javascript routines to control the cameras, which can be used in a different AJAX applications.
 
See [[JavaScript API]]
 
See [[JavaScript API]]
 +
 +
==== Web Interface Ideas ====
 +
There are many new ideas for the web-control-panel. They are being [[Web Interface Ideas|developed]]
 +
 +
==== [[How to obtain an image or a video from an Elphel 353 camera?]] ====
  
 
=== Camera Software ===
 
=== Camera Software ===
 +
 +
==== Axis based SDK for GNU/Linux on EtraxFS ====
 +
Our camera has two main CPUs: EtraxFS running GNU/Linux and Xilinx Spartan 3e FPGA.
 +
 +
The firmware of our cameras is based on [http://developer.axis.com/wiki/doku.php?id=axis:sdk, Axis SDK]. Sources & binaries of our firmwares can be downloaded from [http://sourceforge.net/project/showfiles.php?group_id=105686&package_id=222883, SourceForge project].
 +
 +
==== FPGA bitstream ====
 +
Elphel uses Xilinx Spartant 3e FPGA for image/video encoding.
 +
 +
The FPGA code is distributed under [http://www.gnu.org/licenses/gpl.html. GNU/GPL v3 License].
 +
 +
[http://sourceforge.net/project/showfiles.php?group_id=105686&package_id=184538, VDT Eclipse plugin] can be used to edit Verilog code of the camera. Xilinx WebPACK ISE is free but the download is restricted, each end-user should register. That is why we did not include it in our [[ElphelSoftwareKit]], but you can download it from [http://www.xilinx.com/ise/logic_design_prod/webpack.htm, Xilinx web site.]
 +
 +
===== [[FPGA_Development_in_Elphel_cameras]] =====
 +
This section discribes how to setup the environment for [[FPGA_Development_in_Elphel_cameras|FPGA development with Elphel cameras]].
 +
 +
You can use [[KDevelop]] with Icarus Verilog & GTKWave for HDL Projects Simulation.
 +
 +
==== How to hack ? ====
 +
If you intend to modify source code (GNU/Linux or FPGA), download [[ElphelSoftwareKit]] and live-boot, or better yet install it on your (descent) computer.
 +
 +
But here are several things you can hack already with some softwares in the camera:
 +
 +
 +
===== telnetd =====
 +
By default the camera runs telnetd. Login "root", password "pass". You can telnet to the camera and explore the power of a full GNU/Linux distribution.
 +
 +
===== [[fpcf]] =====
 +
 +
You can use fpcf to adjust some FPGA settings. ''use it very carefully !! it may be dangerous and may destroy your hardware.''
 +
 +
===== hello =====
 +
The Hello program is a little script by Andrey to [[Known problems | fix]] a stuck sensor.
 +
 
==== File systems ====
 
==== File systems ====
 
[[333_File_System]]
 
[[333_File_System]]
 +
 +
==== Elphel 353 camera Unicast & Multicast RTSP streamer ====
 +
 +
The Elphel 353 camera is capable of streaming OGM/MJPEG via RTSP on Unicast or Multicast.
 +
 +
The streamer can be tuned via /etc/streamer.conf & /etc/image.conf (for syntax of image.conf have a look to [[Ccam.cgi]])
 +
 +
==== Control Panel ====
 +
The [[Control Panel]] project is a simple page on the cameras to access some commonly used functions, without having to telnet into the camera.
 +
 +
==== JACK audio server for Elphel 353 ====
 +
 +
JACK is system for handling real-time, low latency audio. [http://jackaudio.org/ JACK audio web site.]
 +
=====[[JACK_in_Elphel_cameras| Using JACK in Elphel camera]]=====
  
 
=== Client Software ===
 
=== Client Software ===
 +
 
==== [[MPlayer]] ====
 
==== [[MPlayer]] ====
−
We have MPlayer patched for use with our cameras. Patches for source codes are accessible on Source Forge but compiled package is only for Debian on i386 architecture. We plan to make a compiled packages for PowerPC architecture and also for Slackware.
+
MPlayer is the main player for Elphel cameras. We have patches for MPlayer to make it compatible with a 3-5MPix resolution stream. We are in constant contact with MPlayer's team to integrate our patches to the mainstream. Patches for source code are accessible on our Source Forge project, the compiled package is only for Debian/Ubuntu on i386 architecture.
 +
 
 +
 
 +
==== [[VLC]] ====
 +
 
 +
VLC is a free cross-platform media player. It supports a large number of multimedia formats, without the need for additional codecs. It can also be used as a streaming server, with extended features (video on demand, on the fly transcoding, ...)
 +
 
 +
 
 +
====GSTreamer====
 +
 
 +
GSTreamer is a great free software that can be used with Elphel cameras in many different ways.
 +
 
 +
=====[[Elphel_Camera_Remote_Audio_using_GStreamer|Mux video & remote audio source]]=====
 +
Describe how to use an Elphel 333/353 camera with MJPG in unicast or multicast stream and mux the video with PC's microphone and restream it on internet with a more suitable codec.
 +
 
 +
=====[[Make_a_movie_from_JPEG_images_using_gstreamer|make a movie from JPEG images]]=====
 +
Compress many JPEG images into a OGG/Theora video.
 +
 
 +
=====[[Convert OGM/MJPEG to OGG/Theora]]=====
 +
Transcode OGM/MJPEG video to a OGG/Theora video codec.
 +
 
 +
==== [[Talk:Recorder|Recorder]] ====
 +
We will start a new recorder for a MJPEG RTP stream.
 +
It will be a small standalone program, which will take a multicast IP address and port, receive a stream and record it to files or to stdout.
 +
The most important thing with the recorder is to avoid frame drops (maybe something can be done with network buffering) and control them using timestamps.
 +
Also, the recorder must provide manageable files (under 2GB), but with zero drops between them.
 +
 
 +
==== Plugins for browsers ====
 +
[[GenReS|GenReS plugin]]
  
 
==== [[HTML Video Surveillance]] ====
 
==== [[HTML Video Surveillance]] ====
−
[http://sourceforge.net/project/showfiles.php?group_id=105686&package_id=138717&release_id=358392 Multiple camera view HTML page] is based on [http://sourceforge.net/projects/genres/ GenReS plugin] for [http://www.mozilla.org/ Mozilla/FireFox].
+
[http://sourceforge.net/project/showfiles.php?group_id=105686&package_id=138717&release_id=358392 Multiple camera view HTML page] is based on [[GenReS |GenReS plugin]] for [http://www.mozilla.org/ Mozilla/FireFox].
−
Now works: scrolling by picture dragging (digital PTZ), camera selection, zoom switch, automatic detection of stream stop by timeout.
+
Currently working: scrolling by picture dragging (digital PTZ), camera selection, zoom switch, automatic detection of stream stop by timeout.
−
List of cameras adresses is now editable manually. It will be automaticaly generated in the [[Live CD]].
+
List of camera addresses is now editable manually. It will be automatically generated in the [[Live CD]].
−
The page will runs recording software by user request. Video will be saved to a fixed directory and splitted to separate files by tunable number of frames.
+
The page will run recording software by user request. Video will be saved to a fixed directory and split into separate files with a tunable number of frames.
−
Main parameters of video captire wil be changeable from the page.
+
Main parameters for video capture will be changeable from the page.
−
The page later can be used in the [[#Video Server]].
+
The page can later be used in the [[#Video Server]].
−
==== [[Live CD]] ====
 
−
Elphel live Linux CD contain software for camera users. We also will make a live DVD for developers.
 
−
The live CD based on [http://knopper.net/knoppix/ Knoppix].
 
−
<ul>The software which should be included to a future releases of the Live CD
 
−
<li>[[HTML Video Surveillance]]
 
−
<li>[http://lives.sourceforge.net LiVES] video editor
 
−
<li>client software packages with simple installation for different distributions of GNU/Linux
 
−
</ul>
 
−
Currently we have CD only for i386 architecture.
 
  
−
We have plans to make Live CD for PowerPC too.
+
==== [[ElphelSoftwareKit]] ====
 +
The Elphel Software Kit DVD contains software for camera users and developers.
 +
The live DVD is based on [http://ubuntu.com/ Ubuntu Feisty].
  
−
We should move to DVD distribution as most of the disks are anyway provided with the hardware, not downloaded.
+
Currently we only have a DVD for the i686 architecture.
  
−
The idea of keeping as full Knopix as possible was to introduce GNU/Linux to the camera users who never had this experience before. But these users will get DVD in a box, downloadable CD version can have more standard packages removed and replaced with camera-specific software.
+
The idea of keeping as full a Ubuntu as possible was to introduce GNU/Linux to the camera users who had never had this experience before.
  
−
One of such major additions will be preinstalled camera development environment (possibly based on [http://www.eclipse.org Eclipse]) to simplify modification of the camera code. Again - don't forget that many of those future developers use now only Visual Studio (or how exactly it is called?) and GNU/Linux can be somewhat alien to them. With this environment they might start playing with their code without prior knowledge of GNU/Linux software development process.
+
One of the major additions is a preinstalled camera development environment based on [http://www.eclipse.org Eclipse]) to simplify the modification of the camera code. Again - don't forget that many of those future developers now only use Visual Studio (or whatever it is called?) and GNU/Linux can be somewhat alien to them. This environment allows them to start playing with their code without prior knowledge of the GNU/Linux software development process.
  
−
It can be useful for the hardware/fpga developers too - to be able to write some code to support the hardware features without spending much time on the mastering software development process.
+
It can be useful for hardware/fpga developers too - to be able to write some code to support the hardware features without spending too much time on mastering the software development process.
  
−
=== Video Server ===
+
==== [[353_firmware_upgrade_procedure | 353 Firmware Upgrade Guide]] ====
−
PC-based video server that will archive incoming Ogg Theora incoming streams from several cameras and transcode them on the fly to lower resolution (binary decimation, windowing) and frame rate (i.e. using only key frames) presenting multiple streams (real time and recorded) to the operator. The external interface of the server might be one of the industry standard and compatible with 3-rd party legacy software.
+
 
 +
==== [[Demosaic on client side]] ====
 +
Our actual FPGA code on the camera uses a very simple algorithm to calculate YCbCr from the Bayer pixels. It uses just a 3x3 block of neighbors. The other point is that this algorithm is time consuming and with the 5MPix sensor the FPGA became the bottleneck. So we have added a special [[JP4]] mode that bypasses the Demosaic in the FPGA and provides an image with pixels in each 16x16 macroblock that are rearranged to separate Bayer colors in individual 8x8 blocks, then encoded as monochrome. Demosaic will be applied during post-processing on the host PC. This section describe different algorithms and implementations used to provide this functionality.
 +
 
 +
== [[Network configuration]] ==
 +
This section describe proper network configurations for the camera and the connected computer.
 +
 
 +
It also discuss some network hacks, wireless cards and gateways tested.
  
 
== Camera hardware ==
 
== Camera hardware ==
−
=== RTC ===
+
==== [[353]]/363 series ====
−
[[RTC]]
+
==== Basic Elphel 353 camera ====
−
=== 10331 ===
+
 
−
[[10331]]
+
Basic Elphel 353 Network camera is composed of:
−
===10332 ===
+
* [[10353]] (processor board w/ CPU and FPGA)
−
[[10332]]
+
* [[10338]] (sensor board for Micron MT9P031/MT9P001 5MPix)
−
=== 10334 ===
+
* [[Elphel_camera_parts#NC353L]]
−
[[10334]]
+
 
 +
==== Basic Elphel 363 camera ====
 +
 
 +
* [[10353]] (processor board w/ CPU and FPGA)
 +
* Kodak CCD [[10342]] (CCD Interface Board) with [[10347]]
 +
* standard enclosure for Elphel 363
 +
 
 +
==== Add ons ====
 +
===== [[10359]] (multi-sensor board/multifunction multiplexer) =====
 +
===== [[10357]] (solid state memory) =====
 +
===== [[10349]] (interface board: IDE, USB1.1, RS232, ...) =====
 +
 
 +
==== USB devices tested with Elphel 353 ====
 +
 
 +
Elphel 353 camera with 10349 extension boards have 1 external and 3 internal USB 1.1 connectors.
 +
 
 +
This section of the wiki describe all USB hardware what was tested with Elphel 353 camera.
 +
 
 +
===== Audio =====
 +
===== Blue Tooth =====
 +
 
 +
USB-BlueTooth keys was tested with Elphel 353 (with 10349 board). One of the goal is to create a [[general propose bluetooth remote control]] program written in JAVA for mobiles phones.
 +
 
 +
===== [[Wifi_usb]] =====
 +
 
 +
===== flash memory and hardrive =====
 +
 
 +
Does not make really sens because of the speed of USB1.1, but work. You can use such storage for still images or running more software from the USB disk/flash.
 +
 
 +
===== keyboard & mouse =====
 +
 
 +
==== Other Hardware ====
 +
[[RTC]] (real time clock), [[10331]], [[10332]], [[10334]] (USB lens control)
  
 
== Active Projects ==
 
== Active Projects ==
 +
 +
=== [[HDR images with Elphel cameras]] ===
 +
 +
This section will describe HDR projects build on Elphel cameras by OpenSource Community.
 +
 +
=== [[Alternative IO board for 10353/10363 series]] ===
 +
This page is an open discussion about the evolution of the [[10349]] additional board.
 +
 +
=== Exif data in JPEG images ===
 +
 +
Exif data (Exchangeable image file format) can be included inside each JPEG image. The [[Exif]] section will describe things what are possible to do with Exif data on Elphel cameras.
 +
 
=== Synchronization of the Cameras ===
 
=== Synchronization of the Cameras ===
  
−
Sometimes you need to acquire images triggered by an extarnal event ar several cameras need to be syncronized with each other. [[Camera Synchronization]] is all about it.
+
Sometimes you need to acquire images triggered by an extarnal event or several cameras need to be synchronized with each other. [[Camera Synchronization]] is all about this.
  
 
=== Photo-finish ===
 
=== Photo-finish ===
−
Photo-finish device made of Elphel model 333 camera with additional FPGA code and software - [[Photo-finish]]
+
Photo-finish device made using Elphel model 333 camera with additional FPGA code and software - [[Photo-finish]]
  
 
=== Zeroconf for Elphel cameras ===
 
=== Zeroconf for Elphel cameras ===
 
[[zeroconf for Elphel cameras]]
 
[[zeroconf for Elphel cameras]]
 
=== Elphel cameras and Zoneminder ===
 
=== Elphel cameras and Zoneminder ===
−
We plan to make model 333 camera work with [http://www.zoneminder.com Zoneminder]
+
We plan to make the model 333 camera work with [http://www.zoneminder.com Zoneminder]
 
=== USB host interface ===
 
=== USB host interface ===
  
−
   daugther board with USB and DC-DC power for lens control board [[10334]]
+
   daughter board with USB and DC-DC power for lens control board [[10334]]
 +
 
 +
 
 +
  Proposal for a usb audio solution : [[PCM2903]]
  
 
=== Motorized lens control ===
 
=== Motorized lens control ===
−
I'll try to retrieve what was written before on the motorized lens control. In short - C/CS mount is rather old and does not work well for interchangeable motorized lenses. We are trying to build an adapter from C/CS-mount to a bayonet type connector. And place a tiny 5mm wide PCB ring in that adapter. This [[10331]] PCB has a reprogrammable microcontroller and uses just 2 connections to the camera for power and data signals combined. It provides all the necessary connections for the most types of motorized lenses.  
+
I'll try to retrieve what was written before on the motorized lens control. In short - C/CS mount is rather old and does not work well for interchangeable motorized lenses. We are trying to build an adapter from C/CS-mount to a bayonet type connector. And place a tiny 5mm wide PCB ring in that adapter. This [[10331]] PCB has a reprogrammable microcontroller and uses just 2 connections to the camera for power and data signals combined. It provides all the necessary connections for most types of motorized lenses.  
  
 
   lens control board [[10331]]
 
   lens control board [[10331]]
Line 113: Line 258:
 
Determine working setup
 
Determine working setup
 
-Does the system need a control board  
 
-Does the system need a control board  
−
-CCD board needs longer cable for minimal package when stacking lens on top of board [http://www.maartenmenheere.nl/blog/images/014-001-0.jpg Camera casing]
+
-CCD board needs a longer cable for minimal package when stacking lens on top of board [http://www.maartenmenheere.nl/blog/images/014-001-0.jpg Camera casing]
  
 
'''Step one'''
 
'''Step one'''
Line 127: Line 272:
  
 
Objective: Does it work, at all?
 
Objective: Does it work, at all?
−
Secundary: Battery life? Video quality?
+
Secondary: Battery life? Video quality?
  
 
'''Step two'''
 
'''Step two'''
Line 137: Line 282:
  
 
Objective: Optimize recording setup of video for ease of use
 
Objective: Optimize recording setup of video for ease of use
−
Secundary: optimal settings? correct lens?
+
Secondary: optimal settings? correct lens?
  
 
'''Step three'''
 
'''Step three'''
Line 150: Line 295:
 
[http://www.maartenmenheere.nl/blog/images/014-001-0.jpg Camera casing]
 
[http://www.maartenmenheere.nl/blog/images/014-001-0.jpg Camera casing]
 
[http://www.maartenmenheere.nl/blog/images/camera.jpg Camera casing]
 
[http://www.maartenmenheere.nl/blog/images/camera.jpg Camera casing]
 +
[http://www.maartenmenheere.nl/blog/images/Radjump_elphel.jpg Outdoor enclosure]
 +
[http://www.maartenmenheere.nl/blog/images/CamAssy-Sheet1.pdf Camera casing 2]
  
 
=== Current enclosure design ===
 
=== Current enclosure design ===
Line 155: Line 302:
 
We are switching to extruded aluminum tube (actually original 303/313 also was design for a standard aluminum profile). Model 333 RJ-45 connector is designed to fit into RJField shell [http://www.rjfield.com/ethernet_connectors_rjf_en.htm].
 
We are switching to extruded aluminum tube (actually original 303/313 also was design for a standard aluminum profile). Model 333 RJ-45 connector is designed to fit into RJField shell [http://www.rjfield.com/ethernet_connectors_rjf_en.htm].
  
−
[[Image:Tube_section.jpg]]
+
[[Mechanical]]
 +
 
 +
=== Removal of distortion ===
 +
 
 +
Distortions from the non-instantaneous exposure of the frame can be done in [http://lives.sf.net LiVES]. But first some other infrastructure must be in place:
 +
 
 +
- Camera must start a [http://www.piksel.no/pwiki/VideoJack videojack] server on the host machine, with the correct fps, width, height and frame palette
 +
 
 +
- Camera must activate the videojack receiver in LiVES and connect it the correct videojack server
 +
 
 +
- Camera can start to send unpackaged compressed frames to videojack server, along with an array of floats
 +
 
 +
- Floats will be in pairs for each horizontal band: the vertical compression/expansion (1.0 means no compression) and the horizontal shift (+-shift/width)
 +
 
 +
- LiVES will pull these frames and the float array from the videojack server
 +
 
 +
- LiVES will decode each frame and pass it along with the float array to a [http://www.xs4all.nl/~salsaman/lives/docs/weedspec.txt Weed] effect which will apply the compress/expand/shift
 +
 
 +
- LiVES will receive the altered frame and save it to a stream
 +
==== Post-processing - non realtime ====
 +
CMOS sensor-specific distortions caused by the [[Electronic Rolling Shutter ]] do not need to be removed realtime - the PC power may be not enough to do so. It is perfectly OK to be able to just post-process the video recorded from the camera.
 +
 
 +
--[[User:Andrey.filippov|Andrey.filippov]] 02:39, 9 December 2005 (CST)
 +
 
 +
=== Synchronous lighting ===
 +
 
 +
External light control. [[Synchronous lighting]].
 +
 
 +
=== Auto focus System ===
 +
 
 +
One research team of  EPFL is working about an auto focus with the Elphel 353 camera. A bi dimensional filter will be used to calculate the sharpness of the image and to search the best in focus image.
 +
 
 +
=== How to buy a Canon EOS Lens adapter for Elphel cameras ===
 +
Search for "Kipon Canon EOS Mount Lens to C Mount Body Adapter" [http://www.amazon.com/Kipon-Canon-Mount-Lens-Adapter/dp/B004THUG6I ASIN: B004THUG6I], list price ~$30 USD
 +
 
 +
=== How to build a Canon EOS Lens adapter for Elphel 333 ===
 +
- Canon EOS Lens with adaptator [http://community.elphel.com/pictures/jmfcontrib/jmf_canon_adap1.JPG] [http://community.elphel.com/pictures/jmfcontrib/jmf_canon_adapt2.JPG]
 +
 
 +
A easy way to build a C adapter for Canon EOS Lens [http://community.elphel.com/pictures/jmfcontrib/jmf_333_with_canon_std.JPG].
 +
 
 +
Take a protection of lens, make a central hole and fix two C/CS adapters [http://community.elphel.com/pictures/jmfcontrib/jmf_cs_c.JPG]
 +
to have a good focal distance.
 +
 
 +
We have tested various Canon Lens that professional reporter lend me. [http://community.elphel.com/pictures/jmfcontrib/jmf_prof_lens.JPG]
 +
 
 +
- Canon Lens mounted on 333 : [http://community.elphel.com/pictures/jmfcontrib/IMG_5526.JPG] [http://community.elphel.com/pictures/jmfcontrib/jmf_view_canon17-40.JPG]
 +
 
 +
''Pictures result of various lens :''
 +
 
 +
The main goal of this test is to try to use Canon EOS lens, more reference and less expensive when C Lens.
 +
The second goal is to be able to use hight definition of CMOS sensor without optical resolution of C Lens.
 +
Canon EOS UMS PRO 17-40 : mount on 333 Elphel [http://community.elphel.com/pictures/jmfcontrib/jmf_view_canon17-40.JPG]
 +
 
 +
-- view of Strasbourg House (France) [http://community.elphel.com/pictures/jmfcontrib/17-40pos17equiv80.3bague.ferme.jpg]
 +
 
 +
-- second view with another set of Canon Lens (minimum apperture) [http://community.elphel.com/pictures/jmfcontrib/17-40pos28.2bagus.testlimit.diaph6mm.ferme.jpg]
 +
 
 +
-- view of cat and barbie dole [http://community.elphel.com/pictures/jmfcontrib/objectif50.2bagues.2chat.jpg]
 +
 
 +
In fact : a Canon large vision become a Power Zoom this CMOS size difference. 35mm to 1/2"  [http://community.elphel.com/pictures/jmfcontrib/jmf_mount_canon_333.JPG]
 +
 
 +
Conclusion :
 +
image quality is better than C lens (varifocal computar or fixed-focal Fujinon)
 +
but Canon lens become a super Zoom Lens and we not certify thermal stability of adjustment action (focus)
 +
 
 +
It will be nice to test a Canon fishEyes.
 +
 
 +
Expensive project of Adaptor to use EOS Lens can be find on net [http://www.mir.com.my/rb/photography/hardwares/classics/eos/EF-lenses/index.htm]
 +
or [http://replay.web.archive.org/20090320033850/http://www.birger.com/Merchant2/merchant.mvc?Screen=ef232_home]
 +
 
 +
=== Image motion tracking/measurement system ===
 +
 
 +
Measure image sequence parameters easy (like oscilloscope). [[Scope]].
 +
 
 +
=== Interesting camera links ===
 +
 
 +
Real Time 3D Reconstruction using moving cameras.
 +
Using ERS distortions for dynamic scene interpretation and correction... [http://comsee.univ-bpclermont.fr/public/projects.php]
 +
 
 +
The Rubik’s Cube solving robot [http://www.mechatrons.com/rubotii.htm]
 +
 
 +
=== [[Porting Theora to 353 cameras]] ===
 +
 
 +
Porting the Ogg Theora encoder and streamer from the model 333 camera to the 353.
 +
 
 +
== Known problems ==
 +
 
 +
[[Known problems]]
 +
 
 +
== Diagnostic and repair ==
 +
 
 +
[[Diagnostic and repair]] page in the Cameras Manual
 +
 
 +
=== Changing or restoring the MAC address ===
 +
 
 +
The factory default MAC address of Elphel products is the same as the serial number which begin with 000E64. If you would like to use a MAC address different from the serial number you can edit the /etc/conf.d/mac file.
 +
 
 +
If the serial number has not been configured is possible to set it (but only once!). To set the serial number use the bootblocktool command on the camera:
 +
 
 +
<nowiki>bootblocktool -a SERNO 000E64xxxxxx</nowiki>
 +
 
 +
Exchange “xxxxxx” with the serial number of the board.
 +
 
 +
!As you can do it only once be careful! If you write the MAC address in a wrong format (ie with : separations) the camera will not boot any more.
 +
 
 +
 
 +
You may do it with the special HTML [http://legacy.elphel.com/3fhlo/bootblock/bootblock333.html page] but those serial numbers are supposed to be factory programmed.--[[User:Andrey.filippov|Andrey.filippov]] 05:36, 30 December 2005 (CST)
 +
 
 +
 
 +
== [[Pricing/Discount/Donation Policy]] ==

Latest revision as of 20:22, 16 February 2017

Contents

Background

Elphel was started in 2001 (Magna, UT USA) when I quit my job inspired by the possibilities that Free Software gives to developers (it was not so obvious for my at the time employer). All the projects were covered in LinuxDevices (complete list on the articles is available here). For several years Elphel was a one-man company, in January 2004 I wrote an article Taming of the Iron Penguin (Russian) in the largest Russain computer-related magazine Computerra and announced there a competition among the software developers for the best video streamer to run in the camera. That was a good idea and after the competition itself was over most of the developers remained in the Elphel team. At first - as volunteers, later - as full/part time employees.

Not all of these developers live in Russia - two, including the winner of the competition are from Kiev, Ukraine. But still all of them know Russian much better than English and so most of our technical discussions were on our private Russian-language forum. So far I failed to move these discussions to the broader audience but believe that Wiki technology can help. Here we will mantain most of the site in English but still have some pages/discussions in Russian, translating documents as we go. Or when somebody else needs it and is not satisfied with Babelfish automatic translation. We will try to keep English pages current - anyway even in Elphel not everybody knows Russian.

Please excuse not-so-good English of our developers and feel free to fix the errors if you see them.

--Andrey.filippov 15:34, 22 September 2005 (CST)

Update on 353/363 cameras

There is preliminary documentation for the new model 353/353 cameras hardware here

353/363 cameras use software based on Linux kernel 2.6 (currently - 2.6.19). There will be more changes in software as we used different web server (lighttpd) and included PHP (as fastCGI application).

More information on software will follow, you may already find code here:

Software Architecture of Elphel 3x3 cameras

Software in the Elphel cameras started from Axis Developer Boards Software and was amended for the camera specific functions. It was modified to work with newer hardware (models 303-313/323-333), support more features and now seems to be a good time to make a major redesign instead of applying incremental changes.

Some discussion already started in Russian here - Nc3x3

Related to the architecture are the #Camera Interface and the #Client Software

Elphel will continue developing a web browser based user interface with AJAX technology. That will require developing/modifying player plugins controllable from JavaScript and implementing specific features needed for video surveillance applications - multiple camera views on the same page, digital PTZ (inside the hi-res incoming stream) and temporal decimation (reducing frame rate) that uses as little CPU resources as possible.

The web-based user interface can be especially useful for open hardware as it reduces the entrance threshold for the developer who would like to customize the cameras functionality - regular web development tools are sufficient for the job.

Camera Interface

The camera now has two alternative APIs:

ccam.cgi

The original interface that supports most camera features - ccam.cgi

and

PHP API

Using PHP in Elphel camera
Elphel PHP constants
PHP Examples
Binning

JavaScript library

We will create a set of javascript routines to control the cameras, which can be used in a different AJAX applications. See JavaScript API

Web Interface Ideas

There are many new ideas for the web-control-panel. They are being developed

How to obtain an image or a video from an Elphel 353 camera?

Camera Software

Axis based SDK for GNU/Linux on EtraxFS

Our camera has two main CPUs: EtraxFS running GNU/Linux and Xilinx Spartan 3e FPGA.

The firmware of our cameras is based on Axis SDK. Sources & binaries of our firmwares can be downloaded from SourceForge project.

FPGA bitstream

Elphel uses Xilinx Spartant 3e FPGA for image/video encoding.

The FPGA code is distributed under GNU/GPL v3 License.

VDT Eclipse plugin can be used to edit Verilog code of the camera. Xilinx WebPACK ISE is free but the download is restricted, each end-user should register. That is why we did not include it in our ElphelSoftwareKit, but you can download it from Xilinx web site.

FPGA_Development_in_Elphel_cameras

This section discribes how to setup the environment for FPGA development with Elphel cameras.

You can use KDevelop with Icarus Verilog & GTKWave for HDL Projects Simulation.

How to hack ?

If you intend to modify source code (GNU/Linux or FPGA), download ElphelSoftwareKit and live-boot, or better yet install it on your (descent) computer.

But here are several things you can hack already with some softwares in the camera:


telnetd

By default the camera runs telnetd. Login "root", password "pass". You can telnet to the camera and explore the power of a full GNU/Linux distribution.

fpcf

You can use fpcf to adjust some FPGA settings. use it very carefully !! it may be dangerous and may destroy your hardware.

hello

The Hello program is a little script by Andrey to fix a stuck sensor.

File systems

333_File_System

Elphel 353 camera Unicast & Multicast RTSP streamer

The Elphel 353 camera is capable of streaming OGM/MJPEG via RTSP on Unicast or Multicast.

The streamer can be tuned via /etc/streamer.conf & /etc/image.conf (for syntax of image.conf have a look to Ccam.cgi)

Control Panel

The Control Panel project is a simple page on the cameras to access some commonly used functions, without having to telnet into the camera.

JACK audio server for Elphel 353

JACK is system for handling real-time, low latency audio. JACK audio web site.

Using JACK in Elphel camera

Client Software

MPlayer

MPlayer is the main player for Elphel cameras. We have patches for MPlayer to make it compatible with a 3-5MPix resolution stream. We are in constant contact with MPlayer's team to integrate our patches to the mainstream. Patches for source code are accessible on our Source Forge project, the compiled package is only for Debian/Ubuntu on i386 architecture.


VLC

VLC is a free cross-platform media player. It supports a large number of multimedia formats, without the need for additional codecs. It can also be used as a streaming server, with extended features (video on demand, on the fly transcoding, ...)


GSTreamer

GSTreamer is a great free software that can be used with Elphel cameras in many different ways.

Mux video & remote audio source

Describe how to use an Elphel 333/353 camera with MJPG in unicast or multicast stream and mux the video with PC's microphone and restream it on internet with a more suitable codec.

make a movie from JPEG images

Compress many JPEG images into a OGG/Theora video.

Convert OGM/MJPEG to OGG/Theora

Transcode OGM/MJPEG video to a OGG/Theora video codec.

Recorder

We will start a new recorder for a MJPEG RTP stream. It will be a small standalone program, which will take a multicast IP address and port, receive a stream and record it to files or to stdout. The most important thing with the recorder is to avoid frame drops (maybe something can be done with network buffering) and control them using timestamps. Also, the recorder must provide manageable files (under 2GB), but with zero drops between them.

Plugins for browsers

GenReS plugin

HTML Video Surveillance

Multiple camera view HTML page is based on GenReS plugin for Mozilla/FireFox. Currently working: scrolling by picture dragging (digital PTZ), camera selection, zoom switch, automatic detection of stream stop by timeout. List of camera addresses is now editable manually. It will be automatically generated in the Live CD. The page will run recording software by user request. Video will be saved to a fixed directory and split into separate files with a tunable number of frames. Main parameters for video capture will be changeable from the page. The page can later be used in the #Video Server.

ElphelSoftwareKit

The Elphel Software Kit DVD contains software for camera users and developers. The live DVD is based on Ubuntu Feisty.

Currently we only have a DVD for the i686 architecture.

The idea of keeping as full a Ubuntu as possible was to introduce GNU/Linux to the camera users who had never had this experience before.

One of the major additions is a preinstalled camera development environment based on Eclipse) to simplify the modification of the camera code. Again - don't forget that many of those future developers now only use Visual Studio (or whatever it is called?) and GNU/Linux can be somewhat alien to them. This environment allows them to start playing with their code without prior knowledge of the GNU/Linux software development process.

It can be useful for hardware/fpga developers too - to be able to write some code to support the hardware features without spending too much time on mastering the software development process.

353 Firmware Upgrade Guide

Demosaic on client side

Our actual FPGA code on the camera uses a very simple algorithm to calculate YCbCr from the Bayer pixels. It uses just a 3x3 block of neighbors. The other point is that this algorithm is time consuming and with the 5MPix sensor the FPGA became the bottleneck. So we have added a special JP4 mode that bypasses the Demosaic in the FPGA and provides an image with pixels in each 16x16 macroblock that are rearranged to separate Bayer colors in individual 8x8 blocks, then encoded as monochrome. Demosaic will be applied during post-processing on the host PC. This section describe different algorithms and implementations used to provide this functionality.

Network configuration

This section describe proper network configurations for the camera and the connected computer.

It also discuss some network hacks, wireless cards and gateways tested.

Camera hardware

353/363 series

Basic Elphel 353 camera

Basic Elphel 353 Network camera is composed of:

Basic Elphel 363 camera

  • 10353 (processor board w/ CPU and FPGA)
  • Kodak CCD 10342 (CCD Interface Board) with 10347
  • standard enclosure for Elphel 363

Add ons

10359 (multi-sensor board/multifunction multiplexer)
10357 (solid state memory)
10349 (interface board: IDE, USB1.1, RS232, ...)

USB devices tested with Elphel 353

Elphel 353 camera with 10349 extension boards have 1 external and 3 internal USB 1.1 connectors.

This section of the wiki describe all USB hardware what was tested with Elphel 353 camera.

Audio
Blue Tooth

USB-BlueTooth keys was tested with Elphel 353 (with 10349 board). One of the goal is to create a general propose bluetooth remote control program written in JAVA for mobiles phones.

Wifi_usb
flash memory and hardrive

Does not make really sens because of the speed of USB1.1, but work. You can use such storage for still images or running more software from the USB disk/flash.

keyboard & mouse

Other Hardware

RTC (real time clock), 10331, 10332, 10334 (USB lens control)

Active Projects

HDR images with Elphel cameras

This section will describe HDR projects build on Elphel cameras by OpenSource Community.

Alternative IO board for 10353/10363 series

This page is an open discussion about the evolution of the 10349 additional board.

Exif data in JPEG images

Exif data (Exchangeable image file format) can be included inside each JPEG image. The Exif section will describe things what are possible to do with Exif data on Elphel cameras.

Synchronization of the Cameras

Sometimes you need to acquire images triggered by an extarnal event or several cameras need to be synchronized with each other. Camera Synchronization is all about this.

Photo-finish

Photo-finish device made using Elphel model 333 camera with additional FPGA code and software - Photo-finish

Zeroconf for Elphel cameras

zeroconf for Elphel cameras

Elphel cameras and Zoneminder

We plan to make the model 333 camera work with Zoneminder

USB host interface

 daughter board with USB and DC-DC power for lens control board 10334



 Proposal for a usb audio solution : PCM2903

Motorized lens control

I'll try to retrieve what was written before on the motorized lens control. In short - C/CS mount is rather old and does not work well for interchangeable motorized lenses. We are trying to build an adapter from C/CS-mount to a bayonet type connector. And place a tiny 5mm wide PCB ring in that adapter. This 10331 PCB has a reprogrammable microcontroller and uses just 2 connections to the camera for power and data signals combined. It provides all the necessary connections for most types of motorized lenses. 

 lens control board 10331
 DC-DC power board for motorized lens control board 10332
 lens control board In System Programmer lbcontrol

Outdoor enclosure

Step Zero

Determine working setup -Does the system need a control board -CCD board needs a longer cable for minimal package when stacking lens on top of board Camera casing

Step one Test setup. Assemble all components in a setup that can record video

Components in test setup - Lens (Computar H3Z4512CS varifocal lens? using power) - Elphel USB setup. Is it possible to directly plug in a usb drive. Where does the power come from? - Battery - Usb cable or network calbe - Usb exteral harddrive or flashdrive - ON/off switch

Objective: Does it work, at all? Secondary: Battery life? Video quality?

Step two

Wooden box. Test setup 1 integrated in outside video testing setup.

Components added in test 2 - Hardboard casing

Objective: Optimize recording setup of video for ease of use Secondary: optimal settings? correct lens?

Step three

Building of waterproof casing - Amphenol plugs - Camera window - Casing camera (fibre reinforced composite) - Casing base station (battery + storage) (fibre reinforced composite)

Schematic Camera casing Camera casing Outdoor enclosure Camera casing 2

Current enclosure design

We are switching to extruded aluminum tube (actually original 303/313 also was design for a standard aluminum profile). Model 333 RJ-45 connector is designed to fit into RJField shell [1].

Mechanical

Removal of distortion

Distortions from the non-instantaneous exposure of the frame can be done in LiVES. But first some other infrastructure must be in place:

- Camera must start a videojack server on the host machine, with the correct fps, width, height and frame palette

- Camera must activate the videojack receiver in LiVES and connect it the correct videojack server

- Camera can start to send unpackaged compressed frames to videojack server, along with an array of floats

- Floats will be in pairs for each horizontal band: the vertical compression/expansion (1.0 means no compression) and the horizontal shift (+-shift/width)

- LiVES will pull these frames and the float array from the videojack server

- LiVES will decode each frame and pass it along with the float array to a Weed effect which will apply the compress/expand/shift

- LiVES will receive the altered frame and save it to a stream

Post-processing - non realtime

CMOS sensor-specific distortions caused by the Electronic Rolling Shutter do not need to be removed realtime - the PC power may be not enough to do so. It is perfectly OK to be able to just post-process the video recorded from the camera.

--Andrey.filippov 02:39, 9 December 2005 (CST)

Synchronous lighting

External light control. Synchronous lighting.

Auto focus System

One research team of EPFL is working about an auto focus with the Elphel 353 camera. A bi dimensional filter will be used to calculate the sharpness of the image and to search the best in focus image.

How to buy a Canon EOS Lens adapter for Elphel cameras

Search for "Kipon Canon EOS Mount Lens to C Mount Body Adapter" ASIN: B004THUG6I, list price ~$30 USD

How to build a Canon EOS Lens adapter for Elphel 333

- Canon EOS Lens with adaptator [2] [3]

A easy way to build a C adapter for Canon EOS Lens [4].

Take a protection of lens, make a central hole and fix two C/CS adapters [5] to have a good focal distance.

We have tested various Canon Lens that professional reporter lend me. [6]

- Canon Lens mounted on 333 : [7] [8]

Pictures result of various lens :

The main goal of this test is to try to use Canon EOS lens, more reference and less expensive when C Lens. The second goal is to be able to use hight definition of CMOS sensor without optical resolution of C Lens. Canon EOS UMS PRO 17-40 : mount on 333 Elphel [9]

-- view of Strasbourg House (France) [10]

-- second view with another set of Canon Lens (minimum apperture) [11]

-- view of cat and barbie dole [12]

In fact : a Canon large vision become a Power Zoom this CMOS size difference. 35mm to 1/2" [13]

Conclusion : image quality is better than C lens (varifocal computar or fixed-focal Fujinon) but Canon lens become a super Zoom Lens and we not certify thermal stability of adjustment action (focus)

It will be nice to test a Canon fishEyes.

Expensive project of Adaptor to use EOS Lens can be find on net [14] or [15]

Image motion tracking/measurement system

Measure image sequence parameters easy (like oscilloscope). Scope.

Interesting camera links

Real Time 3D Reconstruction using moving cameras. Using ERS distortions for dynamic scene interpretation and correction... [16]

The Rubik’s Cube solving robot [17]

Porting Theora to 353 cameras

Porting the Ogg Theora encoder and streamer from the model 333 camera to the 353.

Known problems

Known problems

Diagnostic and repair

Diagnostic and repair page in the Cameras Manual

Changing or restoring the MAC address

The factory default MAC address of Elphel products is the same as the serial number which begin with 000E64. If you would like to use a MAC address different from the serial number you can edit the /etc/conf.d/mac file.

If the serial number has not been configured is possible to set it (but only once!). To set the serial number use the bootblocktool command on the camera:

bootblocktool -a SERNO 000E64xxxxxx

Exchange “xxxxxx” with the serial number of the board.

!As you can do it only once be careful! If you write the MAC address in a wrong format (ie with : separations) the camera will not boot any more.


You may do it with the special HTML page but those serial numbers are supposed to be factory programmed.--Andrey.filippov 05:36, 30 December 2005 (CST)


Pricing/Discount/Donation Policy

Retrieved from "https://wiki.elphel.com/index.php?title=Roadmap&oldid=15085"