Difference between revisions of "Eyesis4Pi workflow"

From ElphelWiki
Jump to: navigation, search
(Created page with "==Preface== The following page describes the Eyesis camera post-processing procedures. ==Eyesis Output== thumb|500px|Fig.1 Eyesis frames * 3x HDDs, e...")
 
 
(39 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
==Preface==
+
==Actual Workflow==
−
The following page describes the Eyesis camera post-processing procedures.
 
  
−
==Eyesis Output==
+
===Footage Acquisition===
−
[[Image:Eyesis_frames.jpg|thumb|500px|Fig.1 Eyesis frames]]
+
{| class="wikitable"
−
* 3x HDDs, each containing *.mov files from 3x 3-sensor modules (a single HDD is connected to an individual camera module).
+
|-
−
* Each frame in a *.mov file is a combination of frames from 3 sensors glued vertically - triplet.
+
! Step !! Description !! Time, min !! Comment
−
* The modules are numbered - 1,2,3 - having individual ip address:
+
|-
−
192.168.0.221
+
|align="center"                    | 1
−
192.168.0.222
+
|style="padding:0px 15px 0px 15px;"| Mounting on a vehicle
−
192.168.0.223
+
|align="center"                    | 15
−
* The sub images are [1.1, 1.2, 1.3], [2.1, 2.2, 2.3], [3.1, 3.2, 3.3]. 3.1 is the fisheye image.
+
|style="padding:0px 15px 0px 15px;"| We do not provide racks or backpacks yet
−
* The frames in *.movs are indexed.
+
|-
−
* A single image format is JP4 (a raw format with a higher quality and frame rate against *.jpeg).
+
|align="center"                    | 2
−
* Corresponding frames have the same time stamps generated by the 1st camera.
+
|style="padding:0px 15px 0px 15px;"| Power on and boot
−
* The 1st camera frames are geotagged.
+
|align="center"                    | 1-2
 +
|style="padding:0px 15px 0px 15px;"|
 +
|-
 +
|align="center"                    | 3
 +
|style="padding:0px 15px 0px 15px;"| Network setup
 +
|align="center"                    | 0-1
 +
|style="padding:0px 15px 0px 15px;"| Needed only once. [http://linuxlibrary.org/kubuntu-network-configuration/  Kubuntu Network Configuration].
 +
|-
 +
|align="center"                    | 4
 +
|style="padding:0px 15px 0px 15px;"| [[Eyesis4Pi_User_Guide_-_Recording|Accessing the WebGUI]]  
 +
|align="center"                    | 0-1
 +
|style="padding:0px 15px 0px 15px;"|The camera is controlled through a WebGUI
 +
|-
 +
|align="center"                    | 5
 +
|style="padding:0px 15px 0px 15px;"| System tests
 +
|align="center"                    | 0-1  
 +
|style="padding:0px 15px 0px 15px;"| Optional
 +
|-
 +
|align="center"                    | 6
 +
|style="padding:0px 15px 0px 15px;"| Waiting for GPS lock
 +
|align="center"                    | 0-10
 +
|style="padding:0px 15px 0px 15px;"| Cold start can take up to 10 min
 +
|-
 +
|align="center"                    | 7
 +
|style="padding:0px 15px 0px 15px;"| Setting up paths and shooting parameters
 +
|align="center"                    | 0-1
 +
|style="padding:0px 15px 0px 15px;"| Initial settings can take longer time
 +
|-
 +
|align="center"                    | 8
 +
|style="padding:0px 15px 0px 15px;"| Start recording
 +
|align="center"                    | -
 +
|style="padding:0px 15px 0px 15px;"|
 +
|-
 +
|align="center"                    | 9
 +
|style="padding:0px 15px 0px 15px;"| Stop recording
 +
|align="center"                    | -
 +
|style="padding:0px 15px 0px 15px;"| Output:
 +
* Images in a [[JP4|JP4]] format (a RAW format with a better quality than JPEG)
 +
* Binary log with the IMU,GPS and Internal Timestamp records.
 +
|-
 +
|align="center"                    | 10
 +
|style="padding:0px 15px 0px 15px;"| Downloading GPS/IMU log
 +
|align="center"                    | 0-10
 +
|style="padding:0px 15px 0px 15px;"| Download onto recording PC, depends on the file size.
 +
|-
 +
|align="center"                    | 11
 +
|style="padding:0px 15px 0px 15px;"| Camera power off
 +
|align="center"                    | 0-1
 +
|style="padding:0px 15px 0px 15px;"| Power off the switch, unplug cables
 +
|-
 +
|align="center"                    | 12
 +
|style="padding:0px 15px 0px 15px;"| Taking the camera off the vehicle
 +
|align="center"                    | 10
 +
|style="padding:0px 15px 0px 15px;"| If needed
 +
|}
  
−
==Post-Processing Setup==
+
===Post-Processing===
−
* HDDs are put into a processing PC and the data is copied.
+
{| class="wikitable"
 +
|-
 +
! Step !! Description !! Time, min !! Program !! Comment
 +
|-
 +
|align="center"                    | 1
 +
|style="padding:0px 15px 0px 15px;"| Copying footage from SSDs to the PC
 +
|align="center"                    | 10-30
 +
|align="center" style="padding:0px 15px 0px 15px;"| Manual
 +
|style="padding:0px 15px 0px 15px;"|
 +
|-
 +
|align="center"                    | 2
 +
|style="padding:0px 15px 0px 15px;"| Selecting images for post-processing
 +
|align="center"                    | -
 +
|align="center" style="padding:0px 15px 0px 15px;"| WebGUI - [[Eyesis4Pi_Panorama_Previewer|Panorama Previewer]]
 +
|style="padding:0px 15px 0px 15px;"| Optional, if not all the images are needed
 +
|-
 +
|align="center"                    | 3
 +
|style="padding:0px 15px 0px 15px;"| Copying selected or all the images for post-processing
 +
|align="center"                    | 10-30
 +
|align="center" style="padding:0px 15px 0px 15px;"| WebGUI - [[Eyesis4Pi_Footage_Procedures_page|Footage Procedures]]
 +
|style="padding:0px 15px 0px 15px;"|
 +
|-
 +
|align="center"                    | 4
 +
|style="padding:0px 15px 0px 15px;"| Post-processing in ImageJ
 +
|align="center"                    | 20-30<sup>1</sup>
 +
|align="center" style="padding:0px 15px 0px 15px;"| [[Eyesis4pi_Post-Processing|ImageJ]]
 +
|style="padding:5px 15px 5px 15px;"| Output: transformed according pixel map images ready for stitching and blending.  
 +
Procedures performed:
 +
* Applying pixel mapping information
 +
* Aberration correction
 +
* Distortion correction
 +
* Vignetting correction
 +
* Color correction
 +
* Denoising
 +
* Sharpening
 +
|-
 +
|align="center"                    | 5
 +
|style="padding:0px 15px 0px 15px;"| Stitching and blending
 +
|align="center"                    | 2-5<sup>1</sup>
 +
|style="padding:0px 15px 0px 15px;"| WebGUI - [[Eyesis4Pi_Footage_Procedures_page|Footage Procedures]]
 +
|style="padding:0px 15px 0px 15px;"| Output panoramas in equirectangular projection
 +
|-
 +
|align="center"                    | 6
 +
|style="padding:0px 15px 0px 15px;"| Preparing panoramas for WebGL Viewer
 +
|align="center"                    | 2-5<sup>1</sup>
 +
|style="padding:0px 15px 0px 15px;"| WebGUI - [[Eyesis4Pi_Footage_Procedures_page|Footage Procedures]]
 +
|style="padding:0px 15px 0px 15px;"| Optional
 +
|}
 +
<sup>1</sup>- per single scene
 +
<br><br>
 +
==GUIs Overview==
  
−
==Workflow==
+
=== WebGUI (recording)===
 +
* PHP+JavaScript+HTML.
 +
* Installed on the laptop or PC used for recording.
  
−
==== Elpehl-Eyesis GUI (Graphical User Interface)====
+
[[Eyesis4Pi_User_Guide|More info...]]
 +
<!--
 +
* [http://wiki.elphel.com/index.php?title=Eyesis_gui elphel-eyesis-gui] - camera GUI  
 +
-->
  
−
Elpehl-Eyesis GUI is installed on the laptop and used for controlling the Eyesis camera.
+
{|
 +
|- valign="top"
 +
|[[File:Recording gui settings.jpeg|thumb|500px|Recording WebGUI, settings]]
 +
|[[File:Recording gui pattern.jpeg|thumb|700px|Recording WebGUI, preview - test pattern]]
 +
|}
  
−
* [http://wiki.elphel.com/index.php?title=Eyesis_gui elphel-eyesis-gui] - camera GUI
+
<!--
 +
* The modules haveindividual IP address:
 +
192.168.0.221-229
 +
* 192.168.0.221 - master camera, sends sync signals to others.
 +
* A single image format is JP4 (a RAW format with a better quality than *.jpeg).
 +
* Corresponding frames have the same time stamps generated by the master camera.
 +
* The master camera frames are geotagged.
 +
* The IMU log is stored on 1 of the 2 Compact Flash cards (non-removable) attached to the master camera.
 +
-->
  
−
==== Elphel Eyesis SDK software (pre-installed), for post-processing:====
+
=== Post-Processing===
 +
 
 +
====A. WebGUI part (PHP + JavaScript + HTML)====
 +
* Copying images before processing.
 +
* Extracting GPS coordinates from images.
 +
* Stitching (after ImageJ).
 +
* Formatting/resizing the results.
 +
[[Eyesis4Pi_Footage_Procedures_page|More info...]]
 +
{|
 +
|[[File:Footage procedures.png|thumb|400px|Footage procedures WebGUI]]
 +
|}
 +
 
 +
====B. ImageJ part (Java)====
 +
* Applying pixel mapping information
 +
* Aberration correction
 +
* Distortion correction
 +
* Vignetting correction
 +
* Color correction
 +
* Denoising
 +
* Sharpening
 +
[[Eyesis4pi_Post-Processing|More info...]]
 +
{|
 +
|- valign="top"
 +
|[[File:Eyesis_corrections_plugin.jpeg|thumb|800px|Eyesis corrections plugin interface]]
 +
|}
 +
 
 +
==Links==
 +
* [[Elphel_Eyesis4Pi|Eyesis4Pi Main Article]]
 +
* [[Eyesis4Pi_workflow|Workflow]]
 +
* [[Eyesis4Pi_User_Guide|User Guide - Recording]]
 +
* [[Eyesis4Pi_Footage_Procedures_page|User Guide - Post-Processing: Footage Procedures]]
 +
* [[Eyesis4pi_Post-Processing|User Guide - Post-processing: ImageJ]]
 +
* [[Eyesis4Pi_data_structure|Data Structure Description]]
 +
* [[Eyesis4Pi_Calibration|Calibration Process]]
 +
 
 +
<!--
 +
<font size='2'>
 
  sudo apt-get install apache2 php5 ufraw dcraw libjpeg-progs imagemagick hugin
 
  sudo apt-get install apache2 php5 ufraw dcraw libjpeg-progs imagemagick hugin
 +
</font>
 
* Apache web server + php
 
* Apache web server + php
 
* [http://wiki.elphel.com/index.php?title=Movie2dng Movie2DNG] - extracting JP4 (JP4 is the Elphel raw image format) images from *.MOVs.
 
* [http://wiki.elphel.com/index.php?title=Movie2dng Movie2DNG] - extracting JP4 (JP4 is the Elphel raw image format) images from *.MOVs.
−
* [[Elphel_Software_Kit_for_Ubuntu#ImageJ_and_Elphel_plugins_for_imageJ|ImageJ]] (by 3rd party) - ImageJ with Elphel plugins (image quality enhancement, color balance) for processing JP4s to create TIFF/JPEG output;
+
* [[Elphel_Software_Kit_for_Ubuntu#ImageJ_and_Elphel_plugins_for_imageJ|ImageJ]] (by 3rd party) - ImageJ with Elphel plugins (image quality enhancement, aberration correction, color balance, stitching) for processing JP4s to create TIFF/JPEG output;  
−
* [http://hugin.sourceforge.net/ Hugin tools] (by 3rd party), with Elphel PHP scripts for automatic work  is used for panorama stitching;  
+
* [http://www.imagemagick.org/script/index.php ImageMagick] #  (by 3rd party), also with a PHP scrip by Elphel is used for converting/modifying panorama JPEGs - scaling for WebGL panorama viewer/editor;
−
* [http://www.imagemagick.org/script/index.php ImageMagick] #  (by 3rd party), also with a PHP scrip by Elphel is used for converting/modifying panorama JPEGs - scaling for WebGL panorama viewer;
 
 
# other pre-installed PHP scripts:
 
# other pre-installed PHP scripts:
 
** exif2kml.php - extract the GPS data from images and put it all to a KML file of the route.
 
** exif2kml.php - extract the GPS data from images and put it all to a KML file of the route.
 
** create_links_xml_file.php - read the KML file and create file with records-links of the route points. It is used in [http://eyesisbox.elphel.net/webgl-pano/geo_tests.html Elphel WebGL Viewer]
 
** create_links_xml_file.php - read the KML file and create file with records-links of the route points. It is used in [http://eyesisbox.elphel.net/webgl-pano/geo_tests.html Elphel WebGL Viewer]
  
−
other:
+
Other:
 
* [http://ufraw.sourceforge.net/ Ufraw] (by 3rd party) - to view/process DNG/TIFF
 
* [http://ufraw.sourceforge.net/ Ufraw] (by 3rd party) - to view/process DNG/TIFF
 
* [http://www.cybercom.net/~dcoffin/dcraw/ Dcraw] (by 3rd party) - for converting JPEGs
 
* [http://www.cybercom.net/~dcoffin/dcraw/ Dcraw] (by 3rd party) - for converting JPEGs
Line 45: Line 201:
 
=====Verify installation=====
 
=====Verify installation=====
 
In a terminal window type & enter commands ("command not found" == not installed):
 
In a terminal window type & enter commands ("command not found" == not installed):
 +
<font size='2'>
 
  movie2dng  
 
  movie2dng  
 
  convert  
 
  convert  
Line 51: Line 208:
 
  cjpeg --help
 
  cjpeg --help
 
  dcraw
 
  dcraw
 +
</font>
 +
== Post processing workflow==
  
−
==== Processing panorama possible steps====
+
* Copy all the *.mov or *.jp4 files to certain locations (folders must have the r/w rights enabled for everyone):
−
 
+
<font size='2'>
−
* A stitching template *.pto should be created in [http://hugin.sourceforge.net/ Hugin] It is created once and reused later for every panorama. Elphel is providing a sample template with the camera (individual for each camera).
+
  /data/footage/<folder>
−
 
+
</font>
−
* Copy all the *.mov files to certain locations (folders must have the r/w rights enabled for everyone):
+
All the php scripts are launched from a single page - [[Eyesis4Pi_Footage_Procedures_page|Footage Procedures]]:
−
Currently:
+
* '''<nowiki>http://127.0.0.1/scripts/split_mov.php?dir=<footage_name></nowiki>''' - (usage TBA) - it applies [http://wiki.elphel.com/index.php?title=Movie2dng Movie2DNG] to all the footage files. The output is JP4 files. (if needed)
−
hdd1: /data/disk-1/<footage_name>/
 
−
  hdd2: /data/disk-2/<footage_name>/
 
−
hdd3: /data/disk-3/<footage_name>/
 
−
 
 
−
* '''<nowiki>http://127.0.0.1/scripts/split_mov.php?dir=<footage_name></nowiki>''' - (usage TBA) - it applies [http://wiki.elphel.com/index.php?title=Movie2dng Movie2DNG] to all the footage files. The output is JP4 files.
 
 
* '''<nowiki>http://127.0.0.1/scripts/exif2kml.php?dir=<footage_name></nowiki>''' - it extracts the GPS data from images and puts it all to a KML file of the route. The output is a KML file.
 
* '''<nowiki>http://127.0.0.1/scripts/exif2kml.php?dir=<footage_name></nowiki>''' - it extracts the GPS data from images and puts it all to a KML file of the route. The output is a KML file.
−
* Launch [http://rsbweb.nih.gov/ij/ ImageJ] and process the files. The output is JPEGs (optionally TIFFs). (This step can be skipped, but some other way of converting files to JPEGs is required); after that run movie2dng+dcraw, for example.
+
* Launch [http://rsbweb.nih.gov/ij/ ImageJ] and process the files. The output are enhanced, stitched JPEGs (optionally TIFFs).
−
* '''<nowiki>http://127.0.0.1/scripts/stitch.php?dir=<dir_name></nowiki>''' - stitch the images processed in ImageJ using [http://hugin.sourceforge.net/ Hugin tools].
 
 
* '''<nowiki>http://127.0.0.1/scripts/convert_images_for_webgl.php?dir=<results_dir></nowiki>''' - rescale results using [http://www.imagemagick.org/script/index.php ImageMagick].
 
* '''<nowiki>http://127.0.0.1/scripts/convert_images_for_webgl.php?dir=<results_dir></nowiki>''' - rescale results using [http://www.imagemagick.org/script/index.php ImageMagick].
 +
-->
 +
[[Category:Eyesis4Pi]]
 +
[[Category:User Guide]]

Latest revision as of 12:20, 21 May 2018

Actual Workflow

Footage Acquisition

Step Description Time, min Comment
1 Mounting on a vehicle 15 We do not provide racks or backpacks yet
2 Power on and boot 1-2
3 Network setup 0-1 Needed only once. Kubuntu Network Configuration.
4 Accessing the WebGUI 0-1 The camera is controlled through a WebGUI
5 System tests 0-1 Optional
6 Waiting for GPS lock 0-10 Cold start can take up to 10 min
7 Setting up paths and shooting parameters 0-1 Initial settings can take longer time
8 Start recording -
9 Stop recording - Output:
  • Images in a JP4 format (a RAW format with a better quality than JPEG)
  • Binary log with the IMU,GPS and Internal Timestamp records.
10 Downloading GPS/IMU log 0-10 Download onto recording PC, depends on the file size.
11 Camera power off 0-1 Power off the switch, unplug cables
12 Taking the camera off the vehicle 10 If needed

Post-Processing

Step Description Time, min Program Comment
1 Copying footage from SSDs to the PC 10-30 Manual
2 Selecting images for post-processing - WebGUI - Panorama Previewer Optional, if not all the images are needed
3 Copying selected or all the images for post-processing 10-30 WebGUI - Footage Procedures
4 Post-processing in ImageJ 20-301 ImageJ Output: transformed according pixel map images ready for stitching and blending.

Procedures performed:

  • Applying pixel mapping information
  • Aberration correction
  • Distortion correction
  • Vignetting correction
  • Color correction
  • Denoising
  • Sharpening
5 Stitching and blending 2-51 WebGUI - Footage Procedures Output panoramas in equirectangular projection
6 Preparing panoramas for WebGL Viewer 2-51 WebGUI - Footage Procedures Optional

1- per single scene

GUIs Overview

WebGUI (recording)

  • PHP+JavaScript+HTML.
  • Installed on the laptop or PC used for recording.

More info...

Recording WebGUI, settings
Recording WebGUI, preview - test pattern


Post-Processing

A. WebGUI part (PHP + JavaScript + HTML)

  • Copying images before processing.
  • Extracting GPS coordinates from images.
  • Stitching (after ImageJ).
  • Formatting/resizing the results.

More info...

Footage procedures WebGUI

B. ImageJ part (Java)

  • Applying pixel mapping information
  • Aberration correction
  • Distortion correction
  • Vignetting correction
  • Color correction
  • Denoising
  • Sharpening

More info...

Eyesis corrections plugin interface

Links

Retrieved from "https://wiki.elphel.com/index.php?title=Eyesis4Pi_workflow&oldid=16018"