Difference between revisions of "Roadmap"
(→Outdoor enclosure) |
|||
| Line 1: | Line 1: | ||
== Background == | == Background == | ||
| − | Elphel was started in 2001 (Magna, UT USA) when I quit my job inspired by the possibilities that Free Software gives to developers (it was not so obvious for my | + | Elphel was started in 2001 (Magna, UT USA) when I quit my job inspired by the possibilities that Free Software gives to developers (it was not so obvious for my at the time employer). All the projects were covered in [http://www.linuxdevices.com LinuxDevices] (complete list on the articles is available [http://www.elphel.com/articles/index.html here]). For several years Elphel was a one-man company, in January 2004 I wrote an article [http://www.computerra.ru/hitech/tech/31862/ Taming of the Iron Penguin (Russian)] in the largest Russain computer-related magazine [http://www.computerra.ru Computerra] and announced there a competition among the software developers for the best video streamer to run in the camera. That was a good idea and after the competition itself was over most of the developers remained in the Elphel team. At first - as volunteers, later - as full/part time employees. |
Not all of these developers live in Russia - two, including the winner of the competition are from Kiev, Ukraine. But still all of them know Russian much better than English and so most of our technical discussions were on our private Russian-language forum. So far I failed to move these discussions to the broader audience but believe that Wiki technology can help. Here we will mantain most of the site in English but still have some pages/discussions in Russian, translating documents as we go. Or when somebody else needs it and is not satisfied with [http://babelfish.altavista.com Babelfish] automatic translation. We will try to keep English pages current - anyway even in Elphel not everybody knows Russian. | Not all of these developers live in Russia - two, including the winner of the competition are from Kiev, Ukraine. But still all of them know Russian much better than English and so most of our technical discussions were on our private Russian-language forum. So far I failed to move these discussions to the broader audience but believe that Wiki technology can help. Here we will mantain most of the site in English but still have some pages/discussions in Russian, translating documents as we go. Or when somebody else needs it and is not satisfied with [http://babelfish.altavista.com Babelfish] automatic translation. We will try to keep English pages current - anyway even in Elphel not everybody knows Russian. | ||
| Line 9: | Line 9: | ||
== Software Architecture of Elphel 3x3 cameras == | == Software Architecture of Elphel 3x3 cameras == | ||
| − | Software in the Elphel cameras started from [http://developer.axis.com/ Axis Developer Boards Software] and was amended for the camera specific functions. It was modified to work with | + | Software in the Elphel cameras started from [http://developer.axis.com/ Axis Developer Boards Software] and was amended for the camera specific functions. It was modified to work with newer hardware (models 303-313/323-333), support more features and now seems to be a good time to make a major redesign instead of applying incremental changes. |
Some discussion already started in Russian here - [[Nc3x3]] | Some discussion already started in Russian here - [[Nc3x3]] | ||
| Line 15: | Line 15: | ||
Related to the architecture are the [[#Camera Interface]] and the [[#Client Software]] | Related to the architecture are the [[#Camera Interface]] and the [[#Client Software]] | ||
| − | Elphel will continue developing web browser based user interface with [http://en.wikipedia.org/wiki/AJAX AJAX] | + | Elphel will continue developing a web browser based user interface with [http://en.wikipedia.org/wiki/AJAX AJAX] technology. That will require developing/modifying player plugins controllable from [http://en.wikipedia.org/wiki/Javascript JavaScript] and implementing specific features needed for video surveillance applications - multiple camera views on the same page, digital PTZ (inside the hi-res incoming stream) and temporal decimation (reducing frame rate) that uses as little CPU resources as possible. |
| − | + | The web-based user interface can be especially useful for open hardware as it reduces the entrance threshold for the developer who would like to customize the cameras functionality - regular web development tools are sufficient for the job. | |
=== Camera Interface === | === Camera Interface === | ||
| − | + | The camera now has two alternative APIs: | |
==== ccam.cgi ==== | ==== ccam.cgi ==== | ||
| − | + | The original interface that supports most camera features - [[ccam.cgi]] | |
and | and | ||
| Line 34: | Line 34: | ||
==== JavaScript library ==== | ==== JavaScript library ==== | ||
| − | We will create a set of javascript routines to control cameras, which can be used in a different AJAX applications. | + | We will create a set of javascript routines to control the cameras, which can be used in a different AJAX applications. |
See [[JavaScript API]] | See [[JavaScript API]] | ||
| Line 43: | Line 43: | ||
=== Client Software === | === Client Software === | ||
==== [[Talk:Recorder|Recorder]] ==== | ==== [[Talk:Recorder|Recorder]] ==== | ||
| − | We start new recorder for MJPEG RTP stream. | + | We will start a new recorder for MJPEG RTP stream. |
| − | It will be small standalone program, which will take multicast IP address and port, receive stream and record it to files or to stdout. | + | It will be small standalone program, which will take a multicast IP address and port, receive a stream and record it to files or to stdout. |
| − | The most important with recorder - avoid frame drops (maybe something can be | + | The most important thing with the recorder - to avoid frame drops (maybe something can be done with network buffering) and control them using timestamps. |
| − | And the recorder | + | And the recorder must provide managable files (under 2GB), but with zero drops between them. |
==== Plugins for browsers ==== | ==== Plugins for browsers ==== | ||
| Line 52: | Line 52: | ||
==== [[MPlayer]] ==== | ==== [[MPlayer]] ==== | ||
| − | We have MPlayer patched for use with our cameras. Patches for source | + | We have MPlayer patched for use with our cameras. Patches for source code are accessible on Source Forge but the compiled package is only for Debian on i386 architecture. We plan to make compiled packages for PowerPC architecture and also for Slackware. |
==== [[HTML Video Surveillance]] ==== | ==== [[HTML Video Surveillance]] ==== | ||
[http://sourceforge.net/project/showfiles.php?group_id=105686&package_id=138717&release_id=358392 Multiple camera view HTML page] is based on [[GenReS |GenReS plugin]] for [http://www.mozilla.org/ Mozilla/FireFox]. | [http://sourceforge.net/project/showfiles.php?group_id=105686&package_id=138717&release_id=358392 Multiple camera view HTML page] is based on [[GenReS |GenReS plugin]] for [http://www.mozilla.org/ Mozilla/FireFox]. | ||
| − | + | Currently working: scrolling by picture dragging (digital PTZ), camera selection, zoom switch, automatic detection of stream stop by timeout. | |
| − | List of | + | List of camera adresses is now editable manually. It will be automaticaly generated in the [[Live CD]]. |
| − | The page will | + | The page will run recording software by user request. Video will be saved to a fixed directory and split into separate files by tunable number of frames. |
| − | Main parameters of video capture | + | Main parameters of video capture will be changeable from the page. |
| − | The page later | + | The page can later be used in the [[#Video Server]]. |
==== [[Live CD]] ==== | ==== [[Live CD]] ==== | ||
| − | Elphel live Linux CD | + | The Elphel live Linux CD contains software for camera users. We also will make a live DVD for developers. |
| − | The live CD based on [http://knopper.net/knoppix/ Knoppix]. | + | The live CD is based on [http://knopper.net/knoppix/ Knoppix]. |
| − | <ul>The software which should be included | + | <ul>The software which should be included in future releases of the Live CD |
<li>[[HTML Video Surveillance]] | <li>[[HTML Video Surveillance]] | ||
<li>[http://lives.sourceforge.net LiVES] video editor | <li>[http://lives.sourceforge.net LiVES] video editor | ||
<li>client software packages with simple installation for different distributions of GNU/Linux | <li>client software packages with simple installation for different distributions of GNU/Linux | ||
</ul> | </ul> | ||
| − | Currently we have | + | Currently we only have CDs for the i386 architecture. |
| − | We have plans to make Live CD for PowerPC too. | + | We have plans to make a Live CD for PowerPC too. |
| − | We should move to DVD distribution as most of the disks are | + | We should move to DVD distribution as most of the disks are already provided with the hardware, not downloaded. |
| − | The idea of keeping as full | + | The idea of keeping as full a Knoppix as possible was to introduce GNU/Linux to the camera users who had never had this experience before. But these users will get a DVD in a box, the downloadable CD version can have more standard packages removed and replaced with camera-specific software. |
| − | One of | + | One of the major additions will be a preinstalled camera development environment (possibly based on [http://www.eclipse.org Eclipse]) to simplify the modification of the camera code. Again - don't forget that many of those future developers now only use Visual Studio (or whatever it is called?) and GNU/Linux can be somewhat alien to them. This environment allows them to start playing with their code without prior knowledge of the GNU/Linux software development process. |
| − | It can be useful for | + | It can be useful for hardware/fpga developers too - to be able to write some code to support the hardware features without spending too much time on mastering the software development process. |
see [[LiveCD Realease Notes]] for schedule | see [[LiveCD Realease Notes]] for schedule | ||
=== Video Server === | === Video Server === | ||
| − | PC-based video server that will archive incoming Ogg Theora incoming streams from several cameras and transcode them on the fly to lower resolution (binary decimation, windowing) and frame rate (i.e. using only key frames) presenting multiple streams (real time and recorded) to the operator. The external interface of the server might be one of the industry standard and compatible with | + | PC-based video server that will archive incoming Ogg Theora incoming streams from several cameras and transcode them on the fly to lower resolution (binary decimation, windowing) and frame rate (i.e. using only key frames) presenting multiple streams (real time and recorded) to the operator. The external interface of the server might be one of the industry standard ones and be compatible with 3rd party legacy software. |
== Camera hardware == | == Camera hardware == | ||
| Line 100: | Line 100: | ||
=== Synchronization of the Cameras === | === Synchronization of the Cameras === | ||
| − | Sometimes you need to acquire images triggered by an extarnal event | + | Sometimes you need to acquire images triggered by an extarnal event or several cameras need to be synchronized with each other. [[Camera Synchronization]] is all about this. |
=== Photo-finish === | === Photo-finish === | ||
| − | Photo-finish device made | + | Photo-finish device made using Elphel model 333 camera with additional FPGA code and software - [[Photo-finish]] |
=== Zeroconf for Elphel cameras === | === Zeroconf for Elphel cameras === | ||
[[zeroconf for Elphel cameras]] | [[zeroconf for Elphel cameras]] | ||
=== Elphel cameras and Zoneminder === | === Elphel cameras and Zoneminder === | ||
| − | We plan to make model 333 camera work with [http://www.zoneminder.com Zoneminder] | + | We plan to make the model 333 camera work with [http://www.zoneminder.com Zoneminder] |
=== USB host interface === | === USB host interface === | ||
| − | + | daughter board with USB and DC-DC power for lens control board [[10334]] | |
| − | Proposal for | + | Proposal for a usb audio solution : [[PCM2903]] |
=== Motorized lens control === | === Motorized lens control === | ||
| − | I'll try to retrieve what was written before on the motorized lens control. In short - C/CS mount is rather old and does not work well for interchangeable motorized lenses. We are trying to build an adapter from C/CS-mount to a bayonet type connector. And place a tiny 5mm wide PCB ring in that adapter. This [[10331]] PCB has a reprogrammable microcontroller and uses just 2 connections to the camera | + | I'll try to retrieve what was written before on the motorized lens control. In short - C/CS mount is rather old and does not work well for interchangeable motorized lenses. We are trying to build an adapter from C/CS-mount to a bayonet type connector. And place a tiny 5mm wide PCB ring in that adapter. This [[10331]] PCB has a reprogrammable microcontroller and uses just 2 connections to the camera for power and data signals combined. It provides all the necessary connections for most types of motorized lenses. |
lens control board [[10331]] | lens control board [[10331]] | ||
| Line 129: | Line 129: | ||
Determine working setup | Determine working setup | ||
-Does the system need a control board | -Does the system need a control board | ||
| − | -CCD board needs longer cable for minimal package when stacking lens on top of board [http://www.maartenmenheere.nl/blog/images/014-001-0.jpg Camera casing] | + | -CCD board needs a longer cable for minimal package when stacking lens on top of board [http://www.maartenmenheere.nl/blog/images/014-001-0.jpg Camera casing] |
'''Step one''' | '''Step one''' | ||
| Line 143: | Line 143: | ||
Objective: Does it work, at all? | Objective: Does it work, at all? | ||
| − | + | Secondary: Battery life? Video quality? | |
'''Step two''' | '''Step two''' | ||
| Line 153: | Line 153: | ||
Objective: Optimize recording setup of video for ease of use | Objective: Optimize recording setup of video for ease of use | ||
| − | + | Secondary: optimal settings? correct lens? | |
'''Step three''' | '''Step three''' | ||
| Line 191: | Line 191: | ||
- LiVES will receive the altered frame and save it to a stream | - LiVES will receive the altered frame and save it to a stream | ||
| − | ==== Post-processing - | + | ==== Post-processing - non realtime ==== |
CMOS sensor-specific distortions caused by the [[Electronic Rolling Shutter ]] do not need to be removed realtime - the PC power may be not enough to do so. It is perfectly OK to be able to just post-process the video recorded from the camera. | CMOS sensor-specific distortions caused by the [[Electronic Rolling Shutter ]] do not need to be removed realtime - the PC power may be not enough to do so. It is perfectly OK to be able to just post-process the video recorded from the camera. | ||
Revision as of 08:50, 29 December 2005
Contents
Background
Elphel was started in 2001 (Magna, UT USA) when I quit my job inspired by the possibilities that Free Software gives to developers (it was not so obvious for my at the time employer). All the projects were covered in LinuxDevices (complete list on the articles is available here). For several years Elphel was a one-man company, in January 2004 I wrote an article Taming of the Iron Penguin (Russian) in the largest Russain computer-related magazine Computerra and announced there a competition among the software developers for the best video streamer to run in the camera. That was a good idea and after the competition itself was over most of the developers remained in the Elphel team. At first - as volunteers, later - as full/part time employees.
Not all of these developers live in Russia - two, including the winner of the competition are from Kiev, Ukraine. But still all of them know Russian much better than English and so most of our technical discussions were on our private Russian-language forum. So far I failed to move these discussions to the broader audience but believe that Wiki technology can help. Here we will mantain most of the site in English but still have some pages/discussions in Russian, translating documents as we go. Or when somebody else needs it and is not satisfied with Babelfish automatic translation. We will try to keep English pages current - anyway even in Elphel not everybody knows Russian.
Please excuse not-so-good English of our developers and feel free to fix the errors if you see them.
--Andrey.filippov 15:34, 22 September 2005 (CST)
Software Architecture of Elphel 3x3 cameras
Software in the Elphel cameras started from Axis Developer Boards Software and was amended for the camera specific functions. It was modified to work with newer hardware (models 303-313/323-333), support more features and now seems to be a good time to make a major redesign instead of applying incremental changes.
Some discussion already started in Russian here - Nc3x3
Related to the architecture are the #Camera Interface and the #Client Software
Elphel will continue developing a web browser based user interface with AJAX technology. That will require developing/modifying player plugins controllable from JavaScript and implementing specific features needed for video surveillance applications - multiple camera views on the same page, digital PTZ (inside the hi-res incoming stream) and temporal decimation (reducing frame rate) that uses as little CPU resources as possible.
The web-based user interface can be especially useful for open hardware as it reduces the entrance threshold for the developer who would like to customize the cameras functionality - regular web development tools are sufficient for the job.
Camera Interface
The camera now has two alternative APIs:
ccam.cgi
The original interface that supports most camera features - ccam.cgi
and
API compatible with Axis cameras
This (AxisAPI) makes Elphel cameras work with some third-party software
JavaScript library
We will create a set of javascript routines to control the cameras, which can be used in a different AJAX applications. See JavaScript API
Camera Software
File systems
Client Software
Recorder
We will start a new recorder for MJPEG RTP stream. It will be small standalone program, which will take a multicast IP address and port, receive a stream and record it to files or to stdout. The most important thing with the recorder - to avoid frame drops (maybe something can be done with network buffering) and control them using timestamps. And the recorder must provide managable files (under 2GB), but with zero drops between them.
Plugins for browsers
MPlayer
We have MPlayer patched for use with our cameras. Patches for source code are accessible on Source Forge but the compiled package is only for Debian on i386 architecture. We plan to make compiled packages for PowerPC architecture and also for Slackware.
HTML Video Surveillance
Multiple camera view HTML page is based on GenReS plugin for Mozilla/FireFox. Currently working: scrolling by picture dragging (digital PTZ), camera selection, zoom switch, automatic detection of stream stop by timeout. List of camera adresses is now editable manually. It will be automaticaly generated in the Live CD. The page will run recording software by user request. Video will be saved to a fixed directory and split into separate files by tunable number of frames. Main parameters of video capture will be changeable from the page. The page can later be used in the #Video Server.
Live CD
The Elphel live Linux CD contains software for camera users. We also will make a live DVD for developers. The live CD is based on Knoppix.
- The software which should be included in future releases of the Live CD
- HTML Video Surveillance
- LiVES video editor
- client software packages with simple installation for different distributions of GNU/Linux
Currently we only have CDs for the i386 architecture.
We have plans to make a Live CD for PowerPC too.
We should move to DVD distribution as most of the disks are already provided with the hardware, not downloaded.
The idea of keeping as full a Knoppix as possible was to introduce GNU/Linux to the camera users who had never had this experience before. But these users will get a DVD in a box, the downloadable CD version can have more standard packages removed and replaced with camera-specific software.
One of the major additions will be a preinstalled camera development environment (possibly based on Eclipse) to simplify the modification of the camera code. Again - don't forget that many of those future developers now only use Visual Studio (or whatever it is called?) and GNU/Linux can be somewhat alien to them. This environment allows them to start playing with their code without prior knowledge of the GNU/Linux software development process.
It can be useful for hardware/fpga developers too - to be able to write some code to support the hardware features without spending too much time on mastering the software development process.
see LiveCD Realease Notes for schedule
Video Server
PC-based video server that will archive incoming Ogg Theora incoming streams from several cameras and transcode them on the fly to lower resolution (binary decimation, windowing) and frame rate (i.e. using only key frames) presenting multiple streams (real time and recorded) to the operator. The external interface of the server might be one of the industry standard ones and be compatible with 3rd party legacy software.
Camera hardware
RTC
10331
10332
10334
Active Projects
Synchronization of the Cameras
Sometimes you need to acquire images triggered by an extarnal event or several cameras need to be synchronized with each other. Camera Synchronization is all about this.
Photo-finish
Photo-finish device made using Elphel model 333 camera with additional FPGA code and software - Photo-finish
Zeroconf for Elphel cameras
Elphel cameras and Zoneminder
We plan to make the model 333 camera work with Zoneminder
USB host interface
daughter board with USB and DC-DC power for lens control board 10334
Proposal for a usb audio solution : PCM2903
Motorized lens control
I'll try to retrieve what was written before on the motorized lens control. In short - C/CS mount is rather old and does not work well for interchangeable motorized lenses. We are trying to build an adapter from C/CS-mount to a bayonet type connector. And place a tiny 5mm wide PCB ring in that adapter. This 10331 PCB has a reprogrammable microcontroller and uses just 2 connections to the camera for power and data signals combined. It provides all the necessary connections for most types of motorized lenses.
lens control board 10331 DC-DC power board for motorized lens control board 10332 lens control board In System Programmer lbcontrol
Outdoor enclosure
Step Zero
Determine working setup -Does the system need a control board -CCD board needs a longer cable for minimal package when stacking lens on top of board Camera casing
Step one Test setup. Assemble all components in a setup that can record video
Components in test setup - Lens (Computar H3Z4512CS varifocal lens? using power) - Elphel USB setup. Is it possible to directly plug in a usb drive. Where does the power come from? - Battery - Usb cable or network calbe - Usb exteral harddrive or flashdrive - ON/off switch
Objective: Does it work, at all? Secondary: Battery life? Video quality?
Step two
Wooden box. Test setup 1 integrated in outside video testing setup.
Components added in test 2 - Hardboard casing
Objective: Optimize recording setup of video for ease of use Secondary: optimal settings? correct lens?
Step three
Building of waterproof casing - Amphenol plugs - Camera window - Casing camera (fibre reinforced composite) - Casing base station (battery + storage) (fibre reinforced composite)
Schematic Camera casing Camera casing Outdoor enclosure
Current enclosure design
We are switching to extruded aluminum tube (actually original 303/313 also was design for a standard aluminum profile). Model 333 RJ-45 connector is designed to fit into RJField shell [1].
Removal of distortion
Distortions from the non-instantaneous exposure of the frame can be done in LiVES. But first some other infrastructure must be in place:
- Camera must start a videojack server on the host machine, with the correct fps, width, height and frame palette
- Camera must activate the videojack receiver in LiVES and connect it the correct videojack server
- Camera can start to send unpackaged compressed frames to videojack server, along with an array of floats
- Floats will be in pairs for each horizontal band: the vertical compression/expansion (1.0 means no compression) and the horizontal shift (+-shift/width)
- LiVES will pull these frames and the float array from the videojack server
- LiVES will decode each frame and pass it along with the float array to a Weed effect which will apply the compress/expand/shift
- LiVES will receive the altered frame and save it to a stream
Post-processing - non realtime
CMOS sensor-specific distortions caused by the Electronic Rolling Shutter do not need to be removed realtime - the PC power may be not enough to do so. It is perfectly OK to be able to just post-process the video recorded from the camera.
--Andrey.filippov 02:39, 9 December 2005 (CST)
