View source for Tiff file format for pre-processed quad-stereo sets

==Image sets==
Example tree of a single set:
 <font size='2'>'''1527256903_350165/'''
 ├── 1527256903_350165.kml
 ├── '''jp4''' (source files directory)
 │   ├── <font color='RoyalBlue'>1527256903_350165_0.jp4</font>
 │   ├── <font color='RoyalBlue'>...</font>
 │   ├── <font color='RoyalBlue'>1527256903_350165_3.jp4</font>
 │   ├── <font color='RoyalBlue'>1527256903_350165_4.jp4</font>
 │   ├── <font color='RoyalBlue'>...</font>
 │   └── <font color='RoyalBlue'>1527256903_350165_7.jp4</font>
 ├── rating.txt
 ├── thumb.jpeg
 └── '''v03''' (model version)
     ├── <font color='Indigo'>1527256903_350165-00-D0.0.jpeg</font>
     ├── <font color='Indigo'>...</font>
     ├── <font color='Indigo'>1527256903_350165-07-D0.0.jpeg</font>
     ├── <font color='OrangeRed'>1527256903_350165.corr-xml</font>
     ├── <font color='Maroon'>1527256903_350165-DSI_COMBO.tiff</font>
     ├── <font color='DarkSlateGray'>1527256903_350165-EXTRINSICS.corr-xml</font>
     ├── <font color='DarkGoldenrod'>1527256903_350165-img1-texture.png</font>
     ├── <font color='DarkGoldenrod'>...</font>
     ├── <font color='DarkGoldenrod'>1527256903_350165-img2001-texture.png</font>
     ├── <font color='DarkGoldenrod'>1527256903_350165-img_infinity-texture.png</font>
     ├── <font color='DarkGoldenrod'>1527256903_350165.mtl</font>
     ├── <font color='DarkGoldenrod'>1527256903_350165.obj</font>
     ├── <font color='DarkGoldenrod'>1527256903_350165.x3d</font>
     └── '''ml''' (directory with processed data files for ML)
         ├── <font color='ForestGreen'>1527256903_350165-ML_DATA-08B-O-FZ0.05-OFFS-2.00000.tiff</font>
         ├── <font color='ForestGreen'>1527256903_350165-ML_DATA-08B-O-FZ0.05-OFFS-1.00000.tiff</font>
         ├── <font color='ForestGreen'>1527256903_350165-ML_DATA-08B-O-FZ0.05-OFFS0.00000.tiff</font>
         ├── <font color='ForestGreen'>1527256903_350165-ML_DATA-08B-O-FZ0.05-OFFS1.00000.tiff</font>
         └── <font color='ForestGreen'>1527256903_350165-ML_DATA-08B-O-FZ0.05-OFFS2.00000.tiff</font></font>
where:
* <font color='RoyalBlue'>'''*.jp4'''</font> - source files, 0..3 - quad stereo camera #1, 4..7 - quad stereo camera #2, and so on if there are more cameras in the system.
* <font color='Indigo'>'''*-D0.0.jpeg'''</font> - disparity = 0, images are undistorted to a distortion polynom common for each image
* <font color='OrangeRed'>'''*.corr-xml'''</font> - ImageJ plugin's settings file?
* <font color='Maroon'>'''*-DSI_COMBO.tiff'''</font> - Disparity Space Image - tiff stack
* <font color='DarkSlateGray'>'''*-EXTRINSICS.corr-xml'''</font> - extrinsic parameters of the multicamera system
* <font color='DarkGoldenrod'>'''*.x3d, *.png'''</font> - X3D format model with textures. The textures are shared with the OBJ format model
* <font color='DarkGoldenrod'>'''*.obj, *.mtl, *.png'''</font> - OBJ format model with textures
* <font color='ForestGreen'>'''*.tiff'''</font> - TIFF stack of pre-processed images for ML

* *.kml, rating.txt, thumb.jpeg - files, related to the online viewer only

==<font color='ForestGreen'>TIFF image stacks for ML</font>==
[[File:Ml tiff stack imagej.jpeg|thumb|400px|Fig.1 Opened in ImageJ, one of the layers]]
[[File:Tiff layers.jpeg|thumb|400px|Fig.2 multi-page TIFF layers (or TIFF Image stack in ImageJ's terms)]]
[[File:Ml tiff stack python values.jpeg|thumb|400px|Fig.3 Decoded 'value'-layers]]
[[File:Ml tiff stack python data layers.jpeg|thumb|500px|Fig.4 Data layers]]
* The TIFF stack format is described in the [https://community.elphel.com/files/presentations/Elphel_TP-CNN_slides.pdf presentation for CVPR2018, pp.19-21]:
** 8 bpp
** layer dimensions 2916x2178
** 5 layers in the stack: ['''diagm-pair''', '''diago-pair''', '''hor-pairs''', '''vert-pairs''', '''other''']
*** '''diagm-pair'''
*** '''diago-pair'''
*** '''hor-pairs'''
*** '''vert-pairs'''
*** '''other''' - encoded values: estimated disparity, estimated+residual disparity, and confidence for the residual disparity
** The layers are tiled - tiles' wxh is read from the header - currently it's 9x9, making it 324x242 tiles total.
** The values in the '''other''' layer are encoded per tile.
** Essentially it’s a normal TIFF image stack with several headers modified:
*** ImageDescription (tag[270]) contains a string with info about number of images, slices
*** tag[50838] – a list of layers names’ lengths
*** tag[50839] – binary string with encoded layers labels and extra info
* The source files are processed using a plugin for ImageJ, the output file for each set is a tiff stack
* There are a few ways to view the stack:
====ImageJ====
ImageJ (or Fiji) - it has a native support for stacks, each stack has a name label stored (along with related xml info) in the ImageJ tiff tags. To read tiff tags in ImageJ, go '''Image > Show Info...'''
====Python====
'''1.''' Use '''imagej_tiff.py''' from [https://git.elphel.com/Elphel/python3-imagej-tiff python3-imagej-tiff] to:
* <font color='DarkGreen'>'''Get tiff tags values (Pillow)'''</font>
* <font color='DarkGreen'>'''Parse Properties xml data stored by ImageJ in the tiff tags'''</font>
* <font color='DarkGreen'>'''Get tiles dimensions from Properties'''</font>
* <font color='DarkGreen'>'''Convert layers to numpy arrays (split into tiles or optionally not split) for further computations and plotting'''</font>
Example (more examples are commented out in the __main__ section of the script):
 <font size='1' style='line-height:0.5;'>'''~$ python3 imagej_tiff.py 1527256903_350165-ML_DATA-08B-O-FZ0.05-OFFS0.00000.tiff'''
 time: 1531344391.7055812
 time: 1531344392.5336654
 TIFF stack labels: ['diagm-pair', 'diago-pair', 'hor-pairs', 'vert-pairs', 'other']
 <?xml version="1.0" ?>
 <properties>
         <ML_OTHER_TARGET>0</ML_OTHER_TARGET>
         <tileWidth>9</tileWidth>
         <disparityRadiusMain>257.22231560274076</disparityRadiusMain>
         <comment_ML_OTHER_GTRUTH_STRENGTH>Offset of the ground truth strength in the &quot;other&quot; layer tile</comment_ML_OTHER_GTRUTH_STRENGTH>
         <data_min>-0.16894744988183344</data_min>
         <comment_intercameraBaseline>Horizontal distance between the main and the auxiliary camera centers (mm). Disparity is specified for the main camera</comment_intercameraBaseline>
         <ML_OTHER_GTRUTH>2</ML_OTHER_GTRUTH>
         <data_max>0.6260986600450271</data_max>
         <disparityRadiusAux>151.5308819757923</disparityRadiusAux>
         <comment_disparityRadiusAux>Side of the square where 4 main camera subcameras are located (mm). Disparity is specified for the main camera</comment_disparityRadiusAux>
         <comment_disparityRadiusMain>Side of the square where 4 main camera subcameras are located (mm)</comment_disparityRadiusMain>
         <comment_dispOffset>Tile target disparity minum ground truth disparity</comment_dispOffset>
         <comment_tileWidth>Square tile size for each 2d correlation, always odd</comment_tileWidth>
         <comment_data_min>Defined only for 8bpp mode - value, corresponding to -127 (-128 is NaN)</comment_data_min>
         <comment_data_max>Defined only for 8bpp mode - value, corresponding to +127 (-128 is NaN)</comment_data_max>
         <comment_ML_OTHER_TARGET>Offset of the target disparity in the &quot;other&quot; layer tile</comment_ML_OTHER_TARGET>
         <VERSION>1.0</VERSION>
         <dispOffset>0.0</dispOffset>
         <comment_ML_OTHER_GTRUTH>Offset of the ground truth disparity in the &quot;other&quot; layer tile</comment_ML_OTHER_GTRUTH>
         <ML_OTHER_GTRUTH_STRENGTH>4</ML_OTHER_GTRUTH_STRENGTH>
         <intercameraBaseline>1256.0</intercameraBaseline>
 </properties>
 Tiles shape: 9x9
 Data min: -0.16894744988183344
 Data max: 0.6260986600450271
 (2178, 2916, 5)
 Stack of images shape: (242, 324, 9, 9, 4)
 time: 1531344392.7290232
 Stack of values shape: (242, 324, 3)
 time: 1531344393.5556033</font>



Upon opening tiff the image will be a numpy array of shape '''(height,width,layer)''' - for further processing it is reshaped to '''(height_in_tiles, width_in_tiles, tile_height, tile_width, layer)''', example:
 <font size='2'>original tiff stack shape: (2178, 2916, 5)
 image data:  (242, 324, 9, 9, 4)
 values data: (242, 324, 3)</font>

[[File:Ml tiff stack python tile packing.jpeg|thumb|400px|Fig.5 Single layer packing (actually, see the CVPR2018 presentation, p.23)]]
'''2.''' Use '''pack_tile.py''' from [https://git.elphel.com/Elphel/python3-imagej-tiff python3-imagej-tiff] to:
* <font color='DarkGreen'>'''Pack image tiles for better structuring for ML'''</font>
 <font size=2>'''# The current packing is hardcoded in the script: (TY,TX,9,9,4) -> (TY,TX,100)'''
 # Get tiles from a tiff stack
 import pack_tile as pile
 packed_tiles = pile.pack(tile)
 # packed_tiles get stacked with '''estimated disparity''' (see '''test_nn_feed.py''')
 packed_tiles = np.dstack((packed_tiles,values[:,:,0]))