shape and shading
DESCRIPTION
Shape and Shading. Koenderink & Van Dorn Chapter 72 of The Visual Neurosciences. Introduction . Observers are interested in Geometry of objects Material identification Light field: the primary and secondary sources of radiation and how radiation pervades space What we have: an image - PowerPoint PPT PresentationTRANSCRIPT
Shape and Shading
Koenderink & Van DornChapter 72 of The Visual Neurosciences
Introduction
• Observers are interested in – Geometry of objects– Material identification– Light field: the primary and secondary
sources of radiation and how radiation pervades space
• What we have: an image• What we are interested in: the scene
Introduction
• Current topic deals with only the static, achromatic, monocular observer
• Ignoring – Observer motion– Object motion– Color– Binocular – Material properties
Introduction
• We want to know how object shape can be determined from monochrome pictures.
The light field
• Light model: rays (particle model)• Radiators
– Primary: luminous– Secondary
• Reflecting• Scattering
The light field
• Volume (ray) density of radiation: the length of all rays crossing a volume, divided by that volume.
• Or the density of photons crossing that volume during a given time interval.
The light field
• Net flux vector:• Number of photons crossing a unit area
per unit time.• Calculated as proportional to cos(),
where is the angle between a surface patch normal and the beam direction.
• Such a definition is appropriate only for small patches, of course.
The light field
• The light passing thru any such circular patch defines a tube.
• Such tubes may be curved.
The light field
• One might also consider the rays of light passing thru a point.
• There are zero such rays.• However, we can define something
called radiance.• <see handbook>
The light field
• Radiant flux: the total energy emitted by a source, or thru some area.
• Irradiance: the radiant flux per unit area (radiant flux density). W/m2
• Radiance: the flux density per solid unit angle. (W/m2)/sr, where sr = A/r2. This is effectively the energy passing thru the solid angle of one steradian.
The light field
• In these terms, the light field is simply the radiance distribution.
• I.e., for any point, we can consider the rays of light that impinges upon it from all directions.
The light field
• Rays– Origins: primary radiators– End: black surfaces
• Air, smoke, fog, etc. may also absorb ray energy but such effects are ignored in this treatment.
Objects in the light field
• Photons are scattered at object surfaces.• Probability of any photon reflecting is a function
of the incident angle and the viewing (or exit) direction.
• The scattered radiance from a point on a surface, to a particular viewing direction, divided by the irradiance from a given direction, of the surface is called the bidirectional reflectance distribution function (BRDF).
• BRDF depends on 4 angular parameters.
Objects in the light field
• BRDFs tend to be complex functions of the 4 angles.
• Many psychophysical studies and computer models ignore this fact.
• A perfectly white surface has a BRDF of 1/ (integral)
Objects in the light field
• Illumination of a matt sphere by directional light– Shading (attitude effect)– Cast shadow– Body shadow
• Disc appearance from the direction of illumination: disc with dark edge
Objects in the light field
• Natural illumination environments– Combination of directional light (solar) and
diffuse. – Approximately uniform hemispherical
diffuse beam.• Very little body shadow• Ground side will be darker (vignetting)
– Ganzfield• Whiteout
Photometric effects
• Levels of scale– Whole scene– Object– Texture
• Incomplete description• Some averaging
– Subtexture• Material properties• Incorporated into BRDF
Photometric effects• Context
– Whole scene• Object
– Texture» Subtexture
• Examples– Shape from shading affected by contours– Shape from shading with illumination direction information
• Scene cues– Specularities– Shadow directions– Degree of diffusion in shadows
Photometric effects
• Texture and illumination cues• Directional illumination
– Texture most apparent near body shadow• Diffuse illumination
– Texture due to cracks and pits– Darkness here not due to attitudinal
mechanisms but vignetting mechanisms.
Photometric effects
• Specularities• Light may be scattered by underlying
substrates for some angles and reflected for others.
• Disappear in diffuse light fields• Directional illumination combined with surface
texture can produce arrays of specularities– Example: shining ripples on a lake– Orange vs. tomato
Photometric effects
• Inter-reflections • Light bounces from one object onto
others.• Ties scene together and indicates
relations between objects.
Photometric effects
• Deviations from Lambertian– Backscatter– Asperity scattering– Translucence
Photometric effects
• Backscatter– Light projected onto textures tends to be reflected
back more to the source.– Why? Shadows are not visible to the source.– Viewed from other angles, shadows decrease
average luminance.– Attitudinal shading is decreased, making object
look flatter.
Photometric effects• Asperity scattering
– Reflections from hair tips– Common on plants and animals– Furs: hairs are ~parallel with surface provide distinct patterns of
specularities.– Asperity occurs when hairs stick straight up.– More light is reflected to the viewer when there are more tips per
unit view angle (as near contours).– Again: Attitudinal shading is decreased, making object look flatter.– Can create light edges.– Together with Lambertian shading, explains
• The existence of odd order filters in V1• The use of line drawings in art.
Photometric effects• Translucence
– Most materials are translucent at the micro scale.– Thus, BRDF concepts do not exactly apply.– Light is not reflected from a point but enters at one point and
exits at a slightly different point.– However, if we consider BRDF as reflection from small
discs, then micro translucence and microstructures within a surface can explain a BRDF.
The structure of pictures• The radiance distribution from any point determines
the set of possible picture that can be taken from that point.
• However, cameras have their own properties.– Dynamic range– Resolution– MTF– Iimited field of view
• Computer screens, film and printers also have their own limitations on dynamic range, resolution etc.
• Not mentioned: luminance response functions
Shape from shading• Definition: invariant under a group of
transformations.• Equivalently: what’s in the scene, rather than
what’s in the image.• In this analysis, viewpoint and other
parameters are held constant.• Illumination varies.
Shape from shading• Concave / convex ambiguities.• Illumination direction / shape confound• Other cues can help resolve these problems.• Only bumps share inter-reflections with the parent
plane.• Only dents have internal inter-reflections.• Many such details may be missing in computer
graphic implementations, affecting the associated psychophysics.
Shape from shading• Cues in collimated light fields
– Body shadow– Textural quality of the body shadow edge– Structure of specularities– Structure of contour edge (serrated or not)
• Orange vs tomato ( same macro shape )
Shape from shading• Cues in diffuse light fields
– Overall contour shape– Textural effects due to vignetting– Shading
• Due to vignetting not attitudinal
• Lambertian is not realistic– Vignetting is not taken into account.– Inter-reflections are not taken into account.
Shape from shading• Shape from Lambertian shading
– Illumination is assumed directional and the same at all points in the scene
– Light returned determines attitude – Surface normals can be considered as unit vectors on the unit sphere
(Gauss maps)
• Gauss map examples– Plane: point on sphere, degenerate– Cylinders and cones: curves on sphere, degenerate– All others: 2D patches on sphere, 1 to 1 locally
• Isophotes– Lines of equal luminance– Correspond to circles on the Gauss sphere
Shape from shading• The nature of non 1 to 1 maps
– Sphere is multiply covered– Think of a cloth with folds– Folds give rise to critical points (min, max, saddle) in the
irradiance field– Folds of the Gauss map correspond to inflections of the
surface, parabolic curves – Such parabolic curves bound the convex, concave and
saddle regions of the surface– The pattern of such parabolic curves can act as a
description of the shape.
Psychophysical results• Little definitive• Extreme stimulus reduction
– Studying images with only one type of cue (say shading)– Doesn’t provoke strong impressions of shape– Examples
• Silhouettes• Line drawings• Indication of shadows and lit areas
– The more cues available, the more subjects agree – Is the real problem interaction?
• Lack of realism– Published work with computer graphics using unknown or un-
described algorithms
Open problems• “Majority of literature is irrelevant due to incomplete description of
the stimuli, extreme stimuli reduction, or invalid paradigms.”• Stimuli must be complex / natural• Control experiments by using complex / natural images and by
varying only a single cue.• Some stimuli are not generic
– Ellipsoids have isophotes and shadow boundaries are planar curves.– No other shapes do.– Degenerate Gauss maps
• Veridicality– Observer’s response is due to two factors
• Idiosyncratic differences• Cues
– These are often lumped together.
Open problems• How to draw theories from computer vision models
for use in human research?– This is difficult because a computer might return a surface
mesh.– What is the human equivalent?
• What about shading in art?– It differs from actual shading yet we understand it.
• Shape impressions of observers need to be recorded rather than simple yes / no forced choices.