forgetit – some store to remember, some store to forget
DESCRIPTION
With growing storage capacities and sinking storage prices, the paradigm of keeping everything is prevailing. However, keeping information accessible, useable and useful goes far beyond purely keeping things, especially in the long run, and entails expenses much larger than just the storage costs. This issue especially applies to content in Content Management Systems where we increasingly face the situation of creating, managing and storing (preserving) multimedia content, which we might never access again due to the pure volume of content. To overcome these issues, we envision the concept of flexible managed forgetting for information that progressively ceases in importance and finally becomes obsolete as well as for redundant information. We will extend TYPO3 with preservation and forgetting. The forgetting will also reduce the user’s cognitive burden for past activities and information in TYPO3 but still allows access if needed. The same as our brain will retrieve details of our past when remembering and getting associations, the approach will provide such means. Within the Seventh Framework Programme for Research (FP7) of the European Union the "ForgetIT" project strives to build a solution for the mentioned problems. The project has a scope of 3 years and TYPO3 has been selected as CMS to build upon as it is Open Source Software and has an open and active community. This talk will give an introduction into digital preservation and why companies can greatly profit from it. The current status of the research project will be demonstrated. An overview of the project can be found on the projects website (of course made with TYPO3): http://www.forgetit-project.eu/TRANSCRIPT
Some store to remember, some store to forget
Søren SchaffsteinCEO of dkd Internet Service GmbHFrankfurt, Germany
About me
What this is all about
The problem
Storage capacity is ever increasingPrices for storage are falling
How large is large?
Size references
A simple text: an average Wikipedia article ≈ 3.78 kB (no markup)
Lots of text: complete Wikipedia ≈ 13.5 GB (text only, no markup)
An average image (12MP) ≈ 1.3 MB (JPG 90% quality; 24bit/pixel)
An average movie stored on Blu-ray Disc ≈ 25.48 GB
1955 – The IBM 355
Capacity: 12 MB
Cost: 6,233.33 USD/MB
3,250 90
✘0
✘0.16 kB
1970 – The IBM 3330
Capacity: 100 MB
Cost: 259.70 USD/MB
3.94 kB27,089 76 0
✘0
✘
1988 – Seagate ST-238
Capacity: 30 MB
Cost: 9.97 USD/MB
102.71 kB8,126 23 0
✘0
✘
2000 – Western Digital WD600AB
Capacity: 60 GB
Cost: 0.00275 USD/MB
16,644,063 4 47,261 2 363.64 MB
2010 – Seagate ST32000542AS
Capacity: 2 TB
Cost: 0.0000450 USD/MB≈ 5 cent/GB
541,798,941 148 1,538,461 76 21.7 GB
2013 – NSA
Capacity: ∞
Cost: free
∞ ∞ ∞ ∞ it’s free :)
✘
Let’s store everything, then!Cool!
Or, maybe not...
There’s a lot more costs
Retrieval
Maintenance
Indexing
Updates
We need to keep our information
Accessible
Usable
Useful
The concept of Memory Buoyancy
Let’s start to forget!
Memory Buoyancy
time
memory
Memory Buoyancy
Memory Buoyancy
A short overview
The ForgetIT Project
ForgetIT project overview
Consortium of 11 partners
Project start was in February 2013
3 years of research & development
http://www.forgetit-project.eu
The ForgetIT project is funded by the EC within the 7th Framework Programme under the objective "Digital Preservation"(GA 600826).
Project Partners 1/2
Centre for Research and Technology Hellas
dkd Internet Service GmbH
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH
Eurix Srl
Gottfried Wilhelm Leibniz Universität Hannover
Project Partners 2/2
IBM Israel - Science and Technology Ltd
Luleå Tekniska Universitet
The Chancellor, Masters and Scholars of the University of Oxford
The University of Edinburgh
The University of Sheffield
Turk Telekomunikasyon AS
Inspiring people to share!
TYPO3 is the CMS used for the organisational use cases
TYPO3 was chosen because it’s Open Source
We want to raise awareness on the matter of preservation
We will publish our modules under open source licenses
ForgetIT core concepts
Managed Forgetting
Synergetic Preservation
Contextualised Remembering
Do you preserve?
What is preservation?
“Preservation — The protection of cultural property through activities that minimize chemical and physical deterioration and damage and that prevent loss of informational content. The primary goal of preservation is to prolong the existence of cultural property.”
Preservation 101
Problems are caused by
storage medium (disks, tapes, DVD, etc.)
Problems are caused by
storage medium (disks, tapes, DVD, etc.)
format of the data
Problems are caused by
storage medium (disks, tapes, DVD, etc.)
format of the data
availability of the software or operating system
possible encryption
Digital Dark Age
“The digital dark age is a possible future situation where it will be difficult or impossible to read historical electronic documents and multimedia, because they have been stored in an obsolete and obscure file format.” Wikipedia
Preserving a website is not trivial
What do want you preserve?
Content only?
Content and Design?
How often? Stock prices vs. Company History page
How do you deal with browser differences?
How do you preserve functionality? E.g. insurance fee calculator
Preservation Value
Preservation Value
~ 5,000 €
Preservation Value
~ 200,000 €
PrivateOrganisational
The ForgetIT Use Cases
A personal use case:How to organise an ever growing picture collection
Personal Preservation
Typical use cases in the daily work with TYPO3-driven company websites.
Organisational Preservation
Organisational Use Cases
Digital Asset Management
Versioning
Archiving a complete Website
Individual genres and their specific requirements
Example: Press Release
An organisational use case
Press Release Example
Elements of a Press Release
text
image
links
documents
Meta information
Presseinformationen Spielwarenmesse
Global Toy Conference Now on Saturday at the Spielwarenmesse
* Customised programme for retailers: “How to get your customer into the shop”* Conference will take place for the 5th time in Nuremberg on 1 February 2014
All around the world, retailers are wondering how they can still get their customers in their shops in the age of the Internet – because competition for the sale of consumer goods online is growing dramatically. With the topic “How to Get Customers into Your Shop – Successful Pricing, Presentation and Selling” the Global Toy Conference of the Spielwarenmesse demonstrates what parameters business owners can adjust for the future. The conference will take place for the first time in the St Petersburg hall in the NCC East on Saturday. The new earlier date means that more international retailers can take advantage of the knowledge on offer at the toy industry's leading trade fair – from 9 a.m. to 4 p.m. on 1 February 2014.
...
Translations
German English
…
Levels of significance
archive valueAction: keep forever
legal valueAction: keep for legal time
Arc
hive
present valueAction: Keep for x days
trigger valueAction: Check significance
Kee
pD
elet
e
media
meta info
media
meta info
Content Management Systemmedia
meta info
copy
move
refer
media
meta info
media asset
meta info
media asset
meta info
etc.
meta info
editablecontent
meta info
structure(code, users,
plugins, extensions,
etc.)
meta info
externalDigital Asset (DAM)
internal
Archive 1
Info Level 2
Info Level 3
media
meta info
media asset
meta info
media asset
meta info
etc.
meta info
editablecontent
meta info
structure(code, users,
plugins, extensions,
etc.
meta info
Info Level 1
(semi)automatic
static
dynamic
Info Level 4, etc.
Output
Archive 2Delete
Archive 1
Info Level 2
Info Level 3
media
meta info
media asset
meta info
media asset
meta info
etc.
meta info
editablecontent
meta info
structure(code, users,
plugins, extensions,
etc.
meta info
Info Level 1
(semi)automatic
static
dynamic
Info Level 4, etc.
Output
Archive 2Delete
Archive 1
Info Level 2
Info Level 3
media
meta info
media asset
meta info
media asset
meta info
etc.
meta info
editablecontent
meta info
structure(code, users,
plugins, extensions,
etc.
meta info
Info Level 1
(semi)automatic
static
dynamic
Info Level 4, etc.
Output
Archive 2Delete
Archive 1
Info Level 2
Info Level 3
media
meta info
media asset
meta info
media asset
meta info
etc.
meta info
editablecontent
meta info
structure(code, users,
plugins, extensions,
etc.
meta info
Info Level 1
(semi)automatic
static
dynamic
Info Level 4, etc.
Output
Archive 2Delete
Archive 1
Info Level 2
Info Level 3
media
meta info
media asset
meta info
media asset
meta info
etc.
meta info
editablecontent
meta info
structure(code, users,
plugins, extensions,
etc.
meta info
Info Level 1
(semi)automatic
static
dynamic
Info Level 4, etc.
Output
Archive 2Delete
Archive 1 Archive 2Delete
L2
L1
L3
L4
L2
L1
L3
L4
T-CM (Todays Content Management) F-CM (Future Content Management)
Retrieve Service
of a press release
The Information Lifecycle
Information Lifecycle
Collect Create Process Publish Analyse Archive
Collect
Create
Process
Publish
Analyse
Archive
Information Lifecycle
Collect Create Process Publish Analyse ArchiveProcess
Annotations
Example Press Release
Annotation (text) Annotation (image)
global toy conference,conference, podium, speaker, lights
A game about forgetting.
Do you remember?
Do you remember the details?
Which ocean was the ForgetIT Team examining?
Do you remember the details?
Which ocean was the ForgetIT Team examining?
Mediterranean Sea
Do you remember the details?
Which ocean was the ForgetIT Team examining?
Mediterranean Sea
How many people of the ForgetIT Team were carrying a bag?
Do you remember the details?
Which ocean was the ForgetIT Team examining?
Mediterranean Sea
How many people of the ForgetIT Team were carrying a bag?
Do you remember the details?
Which ocean was the ForgetIT Team examining?
Mediterranean Sea
How many people of the ForgetIT Team were carrying a bag?
How many barcodes are on the Western Digital WD600AB?
Do you remember the details?
Which ocean was the ForgetIT Team examining?
Mediterranean Sea
How many people of the ForgetIT Team were carrying a bag?
How many barcodes are on the Western Digital WD600AB?
Do you remember the details?
Which ocean was the ForgetIT Team examining?
Mediterranean Sea
How many people of the ForgetIT Team were carrying a bag?
How many barcodes are on the Western Digital WD600AB?
How many pictures in the shoebox image are mostly blue?
Do you remember the details?
Which ocean was the ForgetIT Team examining?
Mediterranean Sea
How many people of the ForgetIT Team were carrying a bag?
How many barcodes are on the Western Digital WD600AB?
How many pictures in the shoebox image are mostly blue?
or how you can participate
Next steps
We’d love to see you participate!
Reflect your thoughts with us
Take our short survey: http://tinyurl.com/forgetit-webarchiving
Tell us your use cases
Join the development of TYPO3 features
Don’t forget them!
We’d love to discuss them with you ... and a beer or two...
Thank you for your attention!
Sources, Books, Images
References
References (Sources) 1/2
Size of Wikipedia (as of 2013-10-04): https://en.wikipedia.org/wiki/Wikipedia:Size_comparisons
Average JPG size: http://web.forret.com/tools/megapixel.asp?title=12+Megapixel+camera&width=4000&height=3000
Average movie size: http://answers.yahoo.com/question/index?qid=20110807095141AABGQm8
Storage Prices: http://www.jcmit.com/diskprice.htm
References (Sources) 2/2
Forget IT Website: http://www.forgetit-project.eu
Preservation: http://unfacilitated.preservation101.org/session1/expl_whatis-definitions.asp
Digital Dark Age: https://en.wikipedia.org/wiki/Digital_dark_age
References (Books)
Delete: The Virtue of Forgetting in the Digital Age, Viktor Mayer-Schönberger
References (Images) 1/8
“About me”: all images by Søren Schaffstein
“ForgetIT Team” by Søren Schaffstein
“The Problem/Knot”: http://www.istockphoto.com/stock-photo-8933647-rope-with-knot.php
“1 Dollar”: http://www.istockphoto.com/stock-photo-17830696-fan-dollars-isolated-on-white.php
Starbucks Cups: http://5feetonagoodday.files.wordpress.com/2012/01/starbucks-coffee-cups-sizes-tall-grande-venti-trenta.jpg
References (Images) 2/8
IBM 355: http://www-03.ibm.com/ibm/history/exhibits/storage/storage_355.html
IBM 3330: http://www-03.ibm.com/ibm/history/exhibits/storage/storage_3330.html
Seagate ST-238: http://www.redlop.de/bilder/produkte/gross/Seagate-WREN-5-ST4702N-702-MB-.png
Western Digital WD600AB: http://www.junek.de/thomas/bilder/WD600AB.jpg
References (Images) 3/8
Seagate ST32000542AS: http://bilder.afterbuy.de/images/ZZNLZ/seagatesata.jpg
Finger “Forget”: http://www.istockphoto.com/stock-photo-7252836-string-finger-reminder-on-white.php
Memory Buoyancy: http://www.istockphoto.com/stock-photo-16244755-fishing-hook-underwater.php?st=0320b45
Fish: http://www.istockphoto.com/stock-photo-14623368-gold-fish-and-piranha.php
References (Images) 4/8
Game pieces by Søren Schaffstein
Managed Forgetting: http://www.istockphoto.com/stock-photo-3533508-colorful-memos.php?st=0320b45
Synergetic Preservation: http://www.istockphoto.com/stock-photo-13301920-goldfish-jump.php
Contextualised Remembering: http://www.istockphoto.com/stock-photo-14370511-shoebox-of-old-photos-too.php
References (Images) 5/8
Cans: http://www.istockphoto.com/stock-photo-16948268-three-metallic-goods-can-with-key.php
5 1/4” Disk: https://secure.flickr.com/photos/twicepix/4330813840/sizes/z/in/photostream/
5 1/4” Disk Drawing: https://secure.flickr.com/photos/flattop341/2094771560/sizes/z/in/photostream/
Ami Pro: http://www.os2museum.com/wp/?attachment_id=99
Digital Dark Age by Søren Schaffstein
References (Images) 6/8
Gauges: http://www.istockphoto.com/stock-photo-9059088-old-gauges.php
Golf Car: http://www.netzeitung.de/default/337276.html#
Golf Car Papers: http://www.motor-talk.de/news/das-heilige-blech-wieder-unterm-hammer-t4421282.html
Create: http://hdwallsize.com/wp-content/uploads/2013/04/Abstract-Art-Wallpaper-Dekstop.jpg
References (Images) 7/8
Process by Søren Schaffstein
Publish: http://www.istockphoto.com/stock-photo-25712828-british-dog-reading.php?st=e5bf164
Analyse: http://www.istockphoto.com/stock-photo-28297160-laboratory-experimental-testing.php?st=239c76e
Archive: http://www.istockphoto.com/stock-photo-18865341-old-wooden-card-catalogue-with-one-opened-drawer.php
References (Images) 8/8
Shoes: http://www.istockphoto.com/stock-photo-2457744-what-s-your-walking-style.php?st=e12d3d2
Questions: http://www.istockphoto.com/stock-photo-17686236-decision-making.php