Orientation fields and 3D shape estimation



Yüklə 33,12 Mb.
tarix30.10.2018
ölçüsü33,12 Mb.
#76228


Orientation fields and 3D shape estimation

  • Roland W. Fleming

  • Max Planck Institute

  • for Biological Cybernetics
















Approach I: inverse optics







Real-world illumination is highly structured

  • Real-world illumination is highly structured

  • Specular reflections of the real world are a bit like texture

  • Can we solve the 3D shape of mirrors using shape-from-texture ?



Image distortions

  • Slant distorts texture but not reflections



Image distortions



Image distortions



Image distortions

  • Curvature distorts reflections but not texture



Curvatures determine distortions



Curvatures determine distortions



Shape-from-texture and shape-from-specularity follow different rules

    • For texture, image compression depends on surface slant
        • first derivative of surface
    • For reflections, image compression depends on surface curvature properties


Local analysis: banding patterns



Gauge Figure Task

  • Subject adjusts 3D orientation of “gauge figure” to match local orientation of surface



Slant and Tilt



Results I



Results II



Is it just the occluding contour?



Interpreting distorted reflections



Population codes



Population codes



Population codes



Population codes





Orientation fields



Orientation fields are robust





Beyond specularity



Differences between diffuse and specular reflection



Differences between diffuse and specular reflection







Beyond specularity



Latent orientation structure



Orientation fields in shading





Reflectance as Illumination







Texture



Texture



Orientation fields in texture







Affine Transformation





Affine Transformation



Affine Transformation



Affine Transformation



Affine Transformation



Illusory distortions of shape



Illusory distortions of shape



Potential of Orientation Fields

  • Uses biologically plausible measurements



Potential of Orientation Fields

  • No need for visual system to estimate reflectance or illumination explicitly.



Potential of Orientation Fields

  • Stable across albedo discontinuities.



Potential of Orientation Fields

  • Handle improbable combinations of reflectance and illumination.



Potential of Orientation Fields

  • May explain how images with no obvious BRDF interpretation nevertheless yield 3D percepts



Converting between cues



Converting between cues



Matte vs. Shiny

  • Same generative statistics, different mappings



Texture vs. Reflectance



Texture vs. Reflectance



Texture vs. Reflectance



Texture vs. Reflectance



Conclusions

  • Orientation fields are potentially a very powerful source of information about 3D shape

  • For the early stages of 3D shape processing, seemingly different cues may have more in common than previously thought





Todd’s Blobs



Todd’s Blobs



What still needs to be explained?

  • For Lambertian materials (or blurry illuminations), the reflectance map is so smooth that it is significantly anisotropic.

  • Therefore shading orientation fields vary considerably with changes in illumination.



What still needs to be explained?



Two possibilities

  • Change in orientation field predicts (subtle) changes in perceived 3D shape

  • There are higher-order invariants in the orientation fields







Synthesis

  • Koenderink, J. J., & van Doorn, A. J. (1980). Photometric invariants related to solid shape. Optica Acta, 27(7), 981-996.

  • Li, A., & Zaidi, Q. (2000). Perception of three-dimensional shape from texture is based on patterns of oriented energy. Vision Research, 40(2), 217-242.

  • Malik, J., & Rosenholtz, R. (1997). Computing local surface orientation and shape from texture for curved surfaces. International Journal of Computer Vision, 23, 149-168.

  • Breton, P., & Zucker, S. W. (1996). Shadows and shading flow fields. In Proceedings of CVPR, 782-789.[[AUTHOR: Please provide the publisher and location for this book.]] San Francisco, CA.

  • Ben-Shahar, O., & Zucker, S. (2001). On the perceptual organization of texture and shading flows: From a geometrical model to coherence computation. In Proceedings of CVPR, pp. 1048-1055.[[AUTHOR: Is this a journal? If so, please provide the volume. If it’s a meeting, please provide the city, state, where the meeting occurred. Thanks.]] Kawaii, HI.

  • Huggins, P. S., Chen, H. F., Belhumeur, P. N., & Zucker, S. W. (2001). Finding folds: On the Appearance and Identification of Occlusion, in CVPR'01. Proceedings IEEE Conference on Computer Vision and Pattern Recognition, 2, 718-725.

  • Grossberg, S. & Mingolla, E. (1987). Neural Dynamics of Surface Perception:Boundary Webs , Illuminants, and Shape from Shading. Commun. Vis. Graph.Image Proc., 37:116—165.



Part II: Conclusions

  • We can accurately and reliably perceive the 3D shape of perfect mirrors

    • Across illuminations
    • In the absence of context to specify the illumination
    • Subjects aren’t just using the occluding boundary to perform the task
  • We think subjects estimate shape directly from the pattern of distortions across the image.

  • The pattern of distortions:

    • can be extracted from the image by populations of simple oriented filters (cf. V1 cell receptive fields)
    • are diagnostic of shape
    • remain quite stable across changes in the illumination




Plastics



When the world is anisotropic



Stripy world



Counter-example I: blurring

  • Blurred versions have sparser edge-maps but still look solid



Counter-example II: negatives

  • Photographic negatives have identical edge-maps but look somewhat flatter



Matte vs. Shiny

  • Same generative statistics, different mappings



Orientation maps



Effects of compression



Shape from texture





Illusory distortions of shape



Yüklə 33,12 Mb.

Dostları ilə paylaş:




Verilənlər bazası müəlliflik hüququ ilə müdafiə olunur ©genderi.org 2024
rəhbərliyinə müraciət

    Ana səhifə