3d processing and metadata ingestion at polimi, gabriele guidi, sara gonizzi barsanti and laura...
DESCRIPTION
3D processing and metadata ingestion at POLIMI, Presentation given by Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli at the 3D ICONS workshop at the XVIII Borso Mediterranea del Turismo Archeologico conference in Paestrum. The presentation describes the 3D digitisation carried out by Politecnico di Milano (POLIMI0 as part of the 3D ICONS project.TRANSCRIPT
![Page 1: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/1.jpg)
3D Processing and metadata
inges1on at POLIMI
Gabriele Guidi*
Sara Gonizzi Barsan1
Laura Loredana Micoli
Politecnico di Milano ‐ Mechanical Engineering Dept.
![Page 2: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/2.jpg)
Project
• 3 years EU-ICT pilot project
• Aim: supply Europeana with 3D items such as:
– Archaeological sites
– Architectures
– Monuments
– Artifacts
Including UNESCO World Heritage assets
http://3dicons-project.eu
![Page 3: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/3.jpg)
Project numbers
• 16 Partners
• 11 Countries
• 3000 3D models + metadata
• 36 months (30 months acquisition phase)
• Project: 100 models/month (average)
• POLIMI unit: 537 3D models + metadata
• POLIMI unit: 18 items/month (average)
Massive digitization project, each production step �
has to be greatly optimized! �
![Page 4: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/4.jpg)
Massive Digitization (Libraries)Book
(physical object)
Bibliographic record (descriptive metadata)
create/convert
2D scan
Digital object
storing
Repository
(technical metadata)
Metadata Record
Digital object url
![Page 5: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/5.jpg)
Massive Digitization (3DICONS)
storing
Repository
(technical metadata)
CH Asset(physical object)
3D capture
3D model(digital object)
(descriptive metadata)
create/convert
Digital object url
Metadata Record
![Page 6: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/6.jpg)
Source of Heritage Assets for POLIMI �The Archaeological Museum in Milan
The architectural structure Settled upon a complex stratification of archeological ruins, tangible sign of the ancient role of Milan as Capital of the Western Roman Empire
The content
1000+ archaeological items including: • epigraphs • statues
• mosaics • furniture
• potteries related to Greek, Etruscan, Roman and Medieval periods
![Page 7: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/7.jpg)
• Specific skill required
The complex ar1cula1on of data requires a high level of
exper1se in the archaeological/historical field
• Time consuming process Collec1ng the informa1on required for arranging
suitable descrip1ve metadata might require more 1me
than allowed by the project dura1on
Descriptive metadata (85%)
Technical metadata definition (1%)
3D acquisition and modeling (14%)
Metadata creation
✗ Only Heritage Assets with pre-existing metadata
have been chosen: conversion instead of creation�
![Page 8: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/8.jpg)
• POLIMI source of metadata is SIRBeC (Information System of Cultural Heritage of the Lombardia Region)
• All records can be exported in xml format
• The SIRBeC data structure is compliant with the CARARE metadata schema, used by 3DICONS as reference for structuring their metadata
• Only metadata mapping has to be designed
Metadata conversion
![Page 9: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/9.jpg)
Data collec1on workflow
![Page 10: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/10.jpg)
3D data collection
Image‐based
modelling
triangula1on‐
based systems TOF system
Small
texturized
objects
Small un‐
texturized
objects
Buildings
(77%)
(14%)
(9%)
![Page 11: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/11.jpg)
SFM• SFM is therefore the most used technology in
this project
• but SFM is nearly a "black box" giving an output with little of no way of intervention on the final output
• the only controllable inputs are good quality images
We need an optimized image acquisition protocol
in order to maximize the quality of 3D output �
![Page 12: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/12.jpg)
Possible imaging problems
• Image blurring due to: – Movement on shooting
– Wrong focusing
– Limited Depth of Field
• Lighting/dynamic range – Backlights/mixed color temp.
– Light spots
– Highlights
• Confusing scene elements – Painted walls/mosaics
– High contrast elements around the subject
![Page 13: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/13.jpg)
DEPTH OF FIELD
![Page 14: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/14.jpg)
Aperture tests on a small artifact
• Camera: Canon 5D mkII
• Sensor: Full frame CMOS 21.1 Mpixel
• Lens: 50mm macro
• Manual focusing on the
left eye @ x10
• Camera-target distance: 22.5 cm
• Avg. GSD: 28µm
15 cm
7.5 cm
![Page 15: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/15.jpg)
Aperture F 2.5
DOF @22.5 cm
2.4 mm
Focal Plane (FP)
FP + 23mm
![Page 16: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/16.jpg)
Aperture F 5.6
DOF @22.5 cm
5.1 mm
FP
FP + 23mm
![Page 17: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/17.jpg)
Aperture F 11
DOF @22.5 cm
10.2 mm
FP
FP + 23mm
![Page 18: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/18.jpg)
Aperture F 22
DOF @22.5 cm
20.4 mm
FP
FP + 23mm
![Page 19: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/19.jpg)
Aperture F 32
DOF @22.5 cm
28.8 mm
FP
FP + 23mm
![Page 20: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/20.jpg)
SFM/matching at different aperturesFront Le^ Right
• Automatic identification of tie points• Image orientation• Dense color cloud image matching (high)
All processing was made with AGISOFT Photoscan
![Page 21: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/21.jpg)
Matchable points vs. aperture
F Tie
points
Dense cloud
size
2.5 200 420637
Focal zone: 22.4 cm – 22.6 cm
![Page 22: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/22.jpg)
Matchable points vs. aperture
F Tie
points
Dense cloud
size
2.5 200 420637
5.6 408 1286552
Focal zone: 22.2 cm – 22.8 cm
![Page 23: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/23.jpg)
Matchable points vs. aperture
F Tie
points
Dense cloud
size
2.5 200 420637
5.6 408 1286552
11 694 2827132
Focal zone: 22 cm – 23 cm
![Page 24: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/24.jpg)
Matchable points vs. aperture
F Tie
points
Dense cloud
size
2.5 200 420637
5.6 408 1286552
11 694 2827132
22 848 3854555
Focal zone: 21.4 cm – 23.6 cm
![Page 25: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/25.jpg)
Matchable points vs. aperture
F Tie
points
Dense cloud
size
2.5 200 420637
5.6 408 1286552
11 694 2827132
22 848 3854555
32 1033 4117599
Focal zone: 21.1 cm – 24.1 cm
![Page 26: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/26.jpg)
LIGHTING/DYNAMIC RANGE
![Page 27: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/27.jpg)
Light filtering on shiny surfaces
No filter
Circular
polarizer
![Page 28: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/28.jpg)
Light filtering
No filter Circular
polarizer
![Page 29: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/29.jpg)
Matchable points vs. filtering
Filtering Tie
points
Dense
cloud size
None 978 1502661
Circular
polarizer
1051 1572751
+5% matched points
![Page 30: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/30.jpg)
HDR processing
• Enhances details in images containing both overexposed and underexposed areas
• Allows therefore to increase the number of points in image matching
• Since the SW manages jpegs only the full HDR is tone mapped and converted in 24 bit RGB
• This allows to increase by the number of matchable points in the darker areas
![Page 31: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/31.jpg)
HDR Processing example
0.60s, f/32Exposure OK
2.50s, f/32+2 stop
0.15s, f/32-2 stop
Tone mappedHDR
• 4 groups of 3 shots have been taken from different orientations
• SFM with:
– Properly exposed shots
– Corresponding one mapped shots
![Page 32: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/32.jpg)
Matchable points vs. processing
Processing Tie
points
Dense
cloud size
None 5467 5637516
HDR 5646 5858700
+4% matched points
![Page 33: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/33.jpg)
CONFUSING SCENE ELEMENTS�
![Page 34: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/34.jpg)
Masking images
Without mask
Withmask
![Page 35: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/35.jpg)
Photo shooting with backgrounds
![Page 36: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/36.jpg)
Corresponding landing page
![Page 37: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/37.jpg)
A few examples - Archaeological Museum, Milan
Texturized mesh models from SFM
![Page 38: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/38.jpg)
Good practices adopted
Issues
• Image blurring due to – Movement on shooting
– Wrong focusing
– Limited Depth of Field
• Lighting/dynamic range – Backlights/mixed color temp.
– Light spots
– Highlights
• Confusing scene elements – Painted walls/mosaics
– High contrast elements around the subject
Shooting/pre-processing solutions
– Tripod
– Manual focusing @ 10x
– Small apertures (16-32)
– Light shielding panels
– Mask post processing
– HDR/Polarizer filter
– Black/white background • hides confusing elements
• speeds up masking
![Page 39: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.vdocuments.mx/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/39.jpg)
Conclusions• The best practices of a massive
3D digitization project has been shown
• Metadata conversion from preexisting sources was needed for quickly generating searchable material
• SFM is a key technology for shortening 3D digitization to a sustainable level
• No many intervention is possible on SFM, the only actual action is improving image quality
• Proper imaging protocols may increase the 3D model quality and the success rate, in possible bad environmental conditions.