Metadata at ICPSR

Sanda Ionescu, ICPSR

Metadata at ICPSR-Catalog Records-

• Created by data processors, who fill out a Web-based form:– Fixed fields,– DDI 2.x and 3.0 compatible

Metadata at ICPSR-Catalog Records-

Metadata at ICPSR-Catalog Records-

• Review and approval by metadata specialist• Stored in ORACLE database• Exported from database to DDI 2.1 XML• XML files stored on server (file system)• HTML and PDF presentation created dynamically

(through XSLT stylesheets) at user request• HTML presentation for viewing only; PDF is

downloadablehttp://www.icpsr.umich.edu/cocoon/ICPSR/STUDY/08589.xml

Metadata at ICPSR-Catalog Records-

• DDI-XML files searched by field from home page to retrieve studies (Inktomisearch)

Metadata at ICPSR-Codebooks-

• HERMES – in-house automated process to generate (most of) the study distribution package:– Input:

• SPSS system or portable• Optional pre-formatted (question) text file

– Output: • Full suite of statistical formats (setups and system)• ASCII data file• DDI 2.1 file with frequencies and question text if available

Metadata at ICPSR-Codebooks-

• DDI 2.1 file may be converted to PDF to generate – An “ICPSR” codebookhttp://www.icpsr.umich.edu/cgi-bin/bob/archive2?study=4699&path=ICPSR&docsonly=yes

– Part of the publicly distributed codebook as other non-DDI resources may be incorporated

http://www.icpsr.umich.edu/cgi-bin/bob/archive2?study=4512&path=ICPSR&docsonly=yes

• In some instances a DDI-based codebook will not be generated

http://www.icpsr.umich.edu/cgi-bin/bob/archive2?study=9522&path=ICPSR&docsonly=yes

Metadata at ICPSR-Codebooks-

• The DDI 2.1 file with variables description– Is archived– Is downloaded into the Social Science Variables

Database (SSVD)

Metadata at ICPSRSocial Science Variables Database

• Also built in ORACLE, but currently a separate entity, with links to studies’ and series’ descriptions.

• Includes variable-level metadata.• Is DDI 2.x and 3.0 compliant (input and output)• Will enable variable-level searches across studies

and series of studies (simple SQL queries - retrieve matches, do not infer relevance)

Integrating DDI 3 into ArchivesSRO-ICPSR collaboration project

Common RELATIONAL DATABASE model for data documentation- Compliant with DDI 3.0 -

Common RELATIONAL DATABASE model for data documentation- Compliant with DDI 3.0 -

Blaiseoutput

Blaiseoutput

SAS/SPSS/Stata files

SAS/SPSS/Stata files DDI 2.xDDI 2.x DDI 3.0DDI 3.0Other…Other…

Client Applications… Web Applications…

SRO: ICPSR:

ICPSR: Variable-level Search

ICPSR projects will be able to use documentation generated by SRO projects…

Metadata at ICPSR-Online analysis-

• Survey Documentation and Analysis (SDA) – Approx. 475 studies – data and documentation in

proprietary format (ddl), DDI 2.x-compatible.– Nesstar - used only as a “test” (currently not in

production mode)https://www.icpsr.umich.edu/ICPSR/access/sda.html

Metadata at ICPSR

• Other study documentation– Questionnaires– User guides– Data definitions

Distributed in machine-readable, but non-searchable formats – PDF, ASCII, Excel, etc.

http://www.icpsr.umich.edu/cgi-bin/bob/archive2?study=4512&path=ICPSR&docsonly=yes