Difference between revisions of "Test Tasks"
m (→2. ERS Correction) |
m |
||
(37 intermediate revisions by the same user not shown) | |||
Line 6: | Line 6: | ||
{| | {| | ||
|valign='top'|[[File:NC353L-369-IMU-GPS 4 comments.jpeg|300px|thumb|IMU, ADIS16375 in NC353L-369 camera]] | |valign='top'|[[File:NC353L-369-IMU-GPS 4 comments.jpeg|300px|thumb|IMU, ADIS16375 in NC353L-369 camera]] | ||
− | |valign='top'|[[File:Eyesis4pi | + | |valign='top'|[[File:Eyesis4pi axes.jpeg|400px|thumb|IMU in Eyesis4π]] |
− | |||
|} | |} | ||
+ | {| | ||
+ | |[[File:Eyesis-with-cradle-pattern_small_2.jpeg|500px|thumb|Fig.3 Positive rotation angles in the calibration machine]] | ||
+ | |[[File:Goniometer.jpeg|500px|thumb|Fig.4 Goniometer]] | ||
+ | |} | ||
+ | {| | ||
+ | |[[File:Eyesis4pi imu accelerations.png|1100px|thumb|Fig.5 IMU acceleration data during the Eyesis4Pi calibration]] | ||
+ | |} | ||
+ | <br/> | ||
+ | ==Event Logger== | ||
+ | {{:Event_logger}} | ||
===Our projects=== | ===Our projects=== | ||
+ | <br/> | ||
====1. Images Relative Orientation==== | ====1. Images Relative Orientation==== | ||
The goal is to find how the camera rotated/moved ([http://en.wikipedia.org/wiki/Rotation_matrix Rotation Matrix]) between the shots. Knowing the images' relative orientation allows to use this information for further 3D reconstruction. | The goal is to find how the camera rotated/moved ([http://en.wikipedia.org/wiki/Rotation_matrix Rotation Matrix]) between the shots. Knowing the images' relative orientation allows to use this information for further 3D reconstruction. | ||
Line 16: | Line 26: | ||
====2. ERS Correction==== | ====2. ERS Correction==== | ||
Our sensor is Aptina's [http://www.aptina.com/products/image_sensors/mt9p006i12stc/ MT9P006] - CMOS, 5MP, 1/2.5". It has an ERS ([http://en.wikipedia.org/wiki/Rolling_shutter Electronic Rolling Shutter]) - the image scan time is about 1/15 s (~66 ms) while IMU samples data at 2.460 kHz (T = 0.406 ms) which makes it possible to reconstruct how the camera moved during the image acquisition and correct the ERS effect by placing each pixel according to the sensor orientation at each moment of time. This only corrects the movement/rotation of the camera but not the ERS effect from the fast moving objects. | Our sensor is Aptina's [http://www.aptina.com/products/image_sensors/mt9p006i12stc/ MT9P006] - CMOS, 5MP, 1/2.5". It has an ERS ([http://en.wikipedia.org/wiki/Rolling_shutter Electronic Rolling Shutter]) - the image scan time is about 1/15 s (~66 ms) while IMU samples data at 2.460 kHz (T = 0.406 ms) which makes it possible to reconstruct how the camera moved during the image acquisition and correct the ERS effect by placing each pixel according to the sensor orientation at each moment of time. This only corrects the movement/rotation of the camera but not the ERS effect from the fast moving objects. | ||
+ | <br/><br/><br/><br/> | ||
+ | ===Test Tasks=== | ||
+ | <br/> | ||
+ | ====Input Data==== | ||
+ | <font size='2'> | ||
+ | * [http://www.analog.com/en/mems-sensors/mems-inertial-measurement-units/adis16375/products/product.html ADIS16375 Datasheet] | ||
+ | * [http://community.elphel.com/files/imu/java/ Reference Java Program] - with an already written parser. | ||
+ | * [http://community.elphel.com/files/imu/00:0E:64:08:1B:89/ Calibration Data] of an Eyesis4π camera: | ||
+ | ** [http://community.elphel.com/files/imu/00:0E:64:08:1B:89/ ''Camera_parameters.csv''] - just to look up some constants that might be needed: lenses parameters, calibration machine (goniometer) parameters. | ||
+ | ** [http://community.elphel.com/files/imu/00:0E:64:08:1B:89/ ''ImageSet_orientation.csv''] | ||
+ | '''timestamp''' - timestamp of an image set taken at certain camera orientation during the calibration process. Seconds since January 1, 1970, the Time Zone is GMT | ||
+ | '''axial''' - angle in degrees about Z-axis (Fig.3) | ||
+ | '''tilt''' - angle in degrees about Y-axis (Fig.3) | ||
+ | '''Motor2''' - motor 2 state in its steps | ||
+ | '''Motor3''' - motor 3 state in its steps | ||
+ | '''NumPoints''' - applies to images only | ||
+ | '''RMS''' - applies to images only | ||
+ | ** Calibration binary log files - [http://community.elphel.com/files/imu/00:0E:64:08:1B:89/ ''test.log-00001...00015'']: | ||
+ | A log file contains 3 types of records (events): | ||
+ | a. (2.460kHz) IMU data | ||
+ | - [localTimeStamp,GMT]: IMU:[gyroX][gyroY][gyroZ][angleX][angleY][angleZ][accelX][accelY][accelZ][veloX][veloY][veloZ][temperature] | ||
+ | where original parameters are angular velocities and accelerations - delta angles and delta velocities represent an integration of original parameters. | ||
+ | b. (5Hz) GPS data | ||
+ | - [localTimeStamp,GMT]: GPS: $GPRMC,231112.2,A,4043.36963,N,11155.90616,W,000.00,089.0,250811,013.2,E | ||
+ | c. (rare) Image-acquired event registered at local clock. | ||
+ | - [localTimeStamp,GMT]: SRC: [masterTimeStamp,GMT][localTimeStamp,GMT], | ||
+ | knowing the master's timestamp is important for matching the events with the taken images - each image has a master timestamp in its EXIF header. | ||
+ | The local timestamp is just the time at which an event was registered. | ||
+ | </font> | ||
+ | <br/><br/> | ||
+ | ====Requirements==== | ||
+ | * Programming language - Java | ||
+ | <br/><br/> | ||
+ | ====<font color='blue'>Task 1.</font> Parse the IMU Data==== | ||
+ | =====Description===== | ||
+ | * The IMU model is [http://www.analog.com/en/mems-sensors/mems-inertial-measurement-units/adis16375/products/product.html ADIS16375]. | ||
+ | * Learn how the log data parser works from the [http://community.elphel.com/files/imu/java/ reference Java program]. | ||
+ | * Modify or write your own parser and plot the IMU data in a spreadsheet as Value(Time) (for accelerations and angular velocities) or Sum(Time) (for delta angles and delta velocities) for a given log file. | ||
+ | |||
+ | =====Comments===== | ||
+ | - | ||
+ | =====Output===== | ||
+ | A spreadsheet with plotted data. | ||
+ | <br/><br/><br/> | ||
+ | ====<font color='blue'>Task 2.</font> Estimating biases (zero drifts) of the IMU Data==== | ||
+ | =====Description===== | ||
+ | * Determine the time intervals of when the camera is at rest. | ||
+ | * Estimate the average biases (zero drifts) of [http://www.analog.com/en/mems-sensors/mems-inertial-measurement-units/adis16375/products/product.html ADIS16375] IMU output data when at rest. The formulas can be retrieved from the [http://community.elphel.com/files/imu/java/ reference program]. | ||
+ | |||
+ | =====Comments===== | ||
+ | * The Camera-IMU system can be considered at rest within some time range from the image registration event - SRC-record in the log file. This range is also to be determined/estimated. | ||
+ | * The first SRC-record is found in [http://community.elphel.com/files/imu/00:0E:64:08:1B:89/ test.log-00002]. | ||
+ | =====Output===== | ||
+ | Estimated zero drifts stored in a public variable and in an XML file. | ||
+ | <br/><br/><br/> | ||
+ | ====<font color='blue'>Task 3.</font> Calculating the IMU parameters more precisely==== | ||
+ | =====Description===== | ||
+ | * Calculate the IMU parameters with a better precision using the [http://community.elphel.com/files/imu/00:0E:64:08:1B:89/ IMU data log files] and the calibration machine orientation log - [http://community.elphel.com/files/imu/00:0E:64:08:1B:89/ ''ImageSet_orientation.csv''] | ||
+ | =====Comments===== | ||
+ | * Calculate the relative IMU rotation matrices for each state at which the pictures were acquired considering the calibration machine angles to be precise enough. | ||
+ | * Knowledge of the precise initial IMU orientation is not needed - can be defined arbitrarily. | ||
+ | * The first SRC-record is found in [http://community.elphel.com/files/imu/00:0E:64:08:1B:89/ test.log-00001]. | ||
+ | * Max records in a log file 10,000,000. | ||
+ | |||
+ | =====Output===== | ||
+ | Calculated with a better precision IMU parameters in a public variable and in an XML file. | ||
+ | <br/><br/> | ||
− | |||
− | |||
− | |||
− | |||
===Links=== | ===Links=== | ||
* [[10338|10338 Sensor board]] | * [[10338|10338 Sensor board]] | ||
+ | * IMU [http://www.analog.com/en/mems-sensors/mems-inertial-measurement-units/adis16488/products/product.html ADIS16488 Datasheet] | ||
+ | * [http://community.elphel.com/files/imu/00:0E:64:08:1B:89/ Calibration Data] for an Eyesis4π camera | ||
+ | * [http://community.elphel.com/files/imu/java/ Reference Java Program] - with built-in log parser | ||
+ | * [http://community.elphel.com/files/imu/read_imu_log.php_txt Log parser written in PHP] | ||
+ | [[Category:IMU]] | ||
+ | [[Category:Event_logger]] |
Latest revision as of 11:49, 23 April 2013
Contents
- 1 Area: IMU Data Processing
- 2 Event Logger
- 2.1 Features
- 2.2 Description
- 2.3 Syncing with an external device (Eyesis4PI - 10353 based)
- 2.4 How to record log on the camera
- 2.5 View recorded logs (read_imu_log.php)
- 2.6 Notes
- 2.7 Examples
- 2.8 Our projects
- 2.9 Test Tasks
- 2.10 Links
Area: IMU Data Processing
About
"An Inertial Measurement Unit (IMU) is an electronic device that measures and reports on a craft's velocity, orientation, and gravitational forces, using a combination of accelerometers and gyroscopes, sometimes also magnetometers..." © Wikipedia
We use ADIS16375 (fully compatible with ADIS16488) with a tri-axis gyroscope and a tri-axis accelerometer.
Event Logger
Features
- ~1 micro second precision
- Up to 4 source channels:
- IMU
- GPS
- Image Acquisition
- External General Purpose Input (e.g., odometer - 3..5V pulses)
Description
The FPGA-based Event Logger uses local clock for time-stamping, so each log entry (IMU, GPS, Image Acquisition and External Input) is recorded with timing info.
In a single camera each acquired image has a timestamp in its header (Exif). The log entry for images has this timestamp recorded at the logger (local) time.
Multiple cameras (e.g., Eyesis4π) are synchronized by the master camera to sub-microsecond, and each acquired also image has the master timestamp in Exif. The log entries for images (if logged in the camera other than master so with different local clock) have 2 fields - master timestamp (same as in image Exif) and local timestamp (same clock as used for IMU), so it is easy to match images with inertial data.
A typical log record has the following format:
[LocalTimeStamp] [SensorData] Examples or parsed records: [LocalTimeStamp]: IMU: [wX] [wY] [wZ] [dAngleX] [dAngleY] [dAngleZ] [accelX] [accelY] [accelZ] [veloX] [veloY] [veloZ] [temperature] [LocalTimeStamp]: GPS: [NMEA sentence] [LocalTimeStamp]: SRC: [MasterTimeStamp]
Syncing with an external device (Eyesis4PI - 10353 based)
An external device (e.g., odometer) can be connected with a camera / camera rig.
The device have to send HTTP requests to be logged to the camera on http://192.168.0.221/imu_setup.php?msg=message_to_log (message is limited to 56 bytes) and 3..5V pulses on the two middle wires of the J15 connector. !! Make sure only the two middle wires are connected and the externals one are not. Since the camera's input trigger is optoisolated, but not the trigger output. !!
For testing purposes we used an Arduino Yún with Adafruit Ultimate GPS Logger Shield. The GPS was used only to send a PPS to the camera's J15 port while Arduino Yún was running a simple script such as:
echo "" > wifi.log ; i=0; while true; do wget http://192.168.0.221/imu_setup.php?msg=$i -O /dev/null -o /dev/null; echo $i ; echo $i >> wifi.log ; iwlist wlan0 scan >> wifi.log ; i=`expr $i + 1`; done
This script is logging an incrementing number both to the camera log and Yún's file system, WiFi scanning is also recorded to the Yún's log. So later both log files can be synchronized in post-processing.
How to record log on the camera
over network
(see http://192.168.0.9/logger_launcher.php source for options and details):
mount_point=/absolute_path - the path at which the storage is mounted (usb or nfs)
from command line (10393 only)
- START:
mkdir /www/pages/logs cat /dev/imu > /www/pages/logs/test.log
- STOP:
CTRL-C or killall cat
View recorded logs (read_imu_log.php)
from 10393s camera
http://192.168.0.9/read_imu_log.php
- looks for logs in /www/pages/logs (automatically created on the first access)
- displays and filters messages - EXT, GPS (NMEA GPVTG, GPGSA, GPGGA, GPRMC), IMU, IMG (4 ports)
- can convert a log to a CSV file (is saved to PC)
Notes
- GPS data is written to both, the event log and the image header (Exif).
- IMU data is written only to the event log.
- Example IMU (ADIS16375) samples rate is 2460Hz.
- Example GPS receiver (Garmin 18x serial) samples rate is 5Hz in NMEA or other configured format.
Examples
Raw
Raw *.log files are found here
Parsed
parsed_log_example.txt (41.3MB) - here
Tools for parsing logs
Download one of the raw logs.
- PHP: Download read_imu_log.php_txt, rename it to *.php and run on a PC with PHP installed.
- Java: Download IMUDataProcessing project for Eclipse.
Our projects
1. Images Relative Orientation
The goal is to find how the camera rotated/moved (Rotation Matrix) between the shots. Knowing the images' relative orientation allows to use this information for further 3D reconstruction.
2. ERS Correction
Our sensor is Aptina's MT9P006 - CMOS, 5MP, 1/2.5". It has an ERS (Electronic Rolling Shutter) - the image scan time is about 1/15 s (~66 ms) while IMU samples data at 2.460 kHz (T = 0.406 ms) which makes it possible to reconstruct how the camera moved during the image acquisition and correct the ERS effect by placing each pixel according to the sensor orientation at each moment of time. This only corrects the movement/rotation of the camera but not the ERS effect from the fast moving objects.
Test Tasks
Input Data
- ADIS16375 Datasheet
- Reference Java Program - with an already written parser.
- Calibration Data of an Eyesis4π camera:
- Camera_parameters.csv - just to look up some constants that might be needed: lenses parameters, calibration machine (goniometer) parameters.
- ImageSet_orientation.csv
timestamp - timestamp of an image set taken at certain camera orientation during the calibration process. Seconds since January 1, 1970, the Time Zone is GMT axial - angle in degrees about Z-axis (Fig.3) tilt - angle in degrees about Y-axis (Fig.3) Motor2 - motor 2 state in its steps Motor3 - motor 3 state in its steps NumPoints - applies to images only RMS - applies to images only
- Calibration binary log files - test.log-00001...00015:
A log file contains 3 types of records (events): a. (2.460kHz) IMU data - [localTimeStamp,GMT]: IMU:[gyroX][gyroY][gyroZ][angleX][angleY][angleZ][accelX][accelY][accelZ][veloX][veloY][veloZ][temperature] where original parameters are angular velocities and accelerations - delta angles and delta velocities represent an integration of original parameters. b. (5Hz) GPS data - [localTimeStamp,GMT]: GPS: $GPRMC,231112.2,A,4043.36963,N,11155.90616,W,000.00,089.0,250811,013.2,E c. (rare) Image-acquired event registered at local clock. - [localTimeStamp,GMT]: SRC: [masterTimeStamp,GMT][localTimeStamp,GMT], knowing the master's timestamp is important for matching the events with the taken images - each image has a master timestamp in its EXIF header. The local timestamp is just the time at which an event was registered.
Requirements
- Programming language - Java
Task 1. Parse the IMU Data
Description
- The IMU model is ADIS16375.
- Learn how the log data parser works from the reference Java program.
- Modify or write your own parser and plot the IMU data in a spreadsheet as Value(Time) (for accelerations and angular velocities) or Sum(Time) (for delta angles and delta velocities) for a given log file.
Comments
-
Output
A spreadsheet with plotted data.
Task 2. Estimating biases (zero drifts) of the IMU Data
Description
- Determine the time intervals of when the camera is at rest.
- Estimate the average biases (zero drifts) of ADIS16375 IMU output data when at rest. The formulas can be retrieved from the reference program.
Comments
- The Camera-IMU system can be considered at rest within some time range from the image registration event - SRC-record in the log file. This range is also to be determined/estimated.
- The first SRC-record is found in test.log-00002.
Output
Estimated zero drifts stored in a public variable and in an XML file.
Task 3. Calculating the IMU parameters more precisely
Description
- Calculate the IMU parameters with a better precision using the IMU data log files and the calibration machine orientation log - ImageSet_orientation.csv
Comments
- Calculate the relative IMU rotation matrices for each state at which the pictures were acquired considering the calibration machine angles to be precise enough.
- Knowledge of the precise initial IMU orientation is not needed - can be defined arbitrarily.
- The first SRC-record is found in test.log-00001.
- Max records in a log file 10,000,000.
Output
Calculated with a better precision IMU parameters in a public variable and in an XML file.
Links
- 10338 Sensor board
- IMU ADIS16488 Datasheet
- Calibration Data for an Eyesis4π camera
- Reference Java Program - with built-in log parser
- Log parser written in PHP