PFSeq : A photorealistic fisheye sequence to evaluate self-calibration, stereo matching, feature points detection and matching, visual odometry, sky segmentation

Targeted applications:

• Fisheye camera intrinsics self-calibration (used with an algorithm needing an initial guess of the field of view, fixed at 180 degrees).
• Fisheye stereo setup extrinsics self-calibration (rotation and translation, up to a scale)
• Stereo matching (with the distance maps)
• Feature points evaluation (using distance maps to recover true matching points)
• Visual odometry (with reference poses)
• Sky segmentation

Download PFSeq:
https://doi.org/10.57745/DYIVVU

This simulated sequence has been created with Blender (using Cycles engine and the Compositor) between 2013 and 2015, in the context of my PhD thesis “Construction of 3D models from fisheye video data—Application to the localisation in urban area”. It depicts an urban canyon, with fisheye cameras facing the sky. It has been motivated by the absence of previous similar data, and by the need of perfect ground truth, that remains an open problem with real acquisition. To reach photorealistic rendering, care has been done on following aspects:

• Buildings’ textures are issued from real photos.
• A blueish area makes the transition of the image circle *.
• Purple fringing issued from chromatic aberrations *.
• Lens flare when the sun is visible *.
• Optical dispersion blur *.
• Motion blur *.

Setup:
Stereo setup with a baseline of 2m along X axis, and a rotational shift on rear camera:

Cameras have a high resolution of 1200×1200 pixels and an effective field of view of 181.8 degrees. Two variants with different fisheye projections:

• equidistant projection: r = f . θ
• equisolid angle projection: r = 2 . f . sin(θ/2)

For each variant, data are available in two versions: with and without realistic optical and motion distortions, depicted with a *.

Data format:
Images are given in lossless RGB png format, sky masks are given in lossless greylevels png format (only 2 colors: black 0 and white 255), distance maps are given in lossless 32 bits floating point exr format (3 equal channels with distance values in metres, and maximum value set to 500.0).

Data organisation:

/equidistant	/back	/distances	Image0000 to Image0075.exr
			/sky		Image0000 to Image0075.png
			/zerodefect	Image0000 to Image0075.png
			Image0000 to Image0075.png
		/front	...
/equisolid	...

Usage example of distance maps (python3):

import numpy as np
import cv2
from matplotlib import pyplot as plt
I = np.asarray(cv2.imread("Image0000.exr", flags=cv2.IMREAD_ANYDEPTH | cv2.IMREAD_ANYCOLOR))[:,:,0]
I[I>500] = 500
plt.imshow(I, cmap='gray', vmin=0, vmax=50)
plt.show()

Usage example of poses (python3):

import numpy as np
import matplotlib.pyplot as plt
poses = np.loadtxt("poses-front.csv", delimiter=";", skiprows=1)
X = poses[:,1]
Y = poses[:,2]
rZ = np.deg2rad(poses[:,3])
vX = np.cos(rZ)
vY = np.sin(rZ)
plt.plot(X, Y, marker='+', color='blue')
plt.quiver(X,Y, vX,vY, color='cyan')
plt.xlabel("X [m]")
plt.ylabel("Y [m]")
plt.show()

License: ODbL-1.0
This PFSeq dataset is made available under the Open Database License: http://opendatacommons.org/licenses/odbl/1.0/. Any rights in individual contents of the database are licensed under the Database Contents License: http://opendatacommons.org/licenses/dbcl/1.0/

If you use this work, please cite:

Julien Moreau, Sébastien Ambellouis, Yassine Ruichek. Feature points evaluation on omnidirectional vision with a photorealistic fisheye sequence – A report on experiments done in 2014. IRTES-SET; IFSTTAR-COSYS-LEOST. 2026. ⟨hal-05492047⟩

BibTeX format:

@techreport{moreau:hal-05492047,
  TITLE = {{Feature points evaluation on omnidirectional vision with a photorealistic fisheye sequence -- A report on experiments done in 2014}},
  AUTHOR = {Moreau, Julien and Ambellouis, Sébastien and Ruichek, Yassine},
  URL = {https://hal.science/hal-05492047},
  INSTITUTION = {{IRTES-SET ; IFSTTAR-COSYS-LEOST}},
  YEAR = {2026},
  MONTH = Feb,
  KEYWORDS = {fisheye ; feature points ; PFSeq ; dataset},
  PDF = {https://hal.science/hal-05492047v1/file/final-report-features-on-fisheye.pdf},
  HAL_ID = {hal-05492047},
  HAL_VERSION = {v1},
}

Acknowledgements

This work has been carried out within the ANR MEDDTL PREDIT CAPLOC project https://temis.documentation.developpement-durable.gouv.fr/document.html?id=Temis-0084273.