“You Must Act” – A Review of the Papers of Abraham Lincoln Introduction The vision for The Papers of Abraham Lincoln (PAL) is to give the world a digital collection of all things written by and to Abraham Lincoln, with images of the documents and annotated transcriptions. We believe that, when this product is finally created, it will transform the way scholars research and write about Abraham Lincoln, and in turn will provide the entire world with a new view of the Great Emancipator. However, after 17 years, PAL’s task remains far from finished, and the work that has been completed has never been made available to the public. Out of concern for the project’s direction and pace, we asked five experts including directors of similar presidential document projects to review PAL. The five members of our Review and Planning Team (RAPT) were:

Daniel Feller: Director of the Papers of Andrew Jackson at the University of Tennessee

Patrick Lewis: Director of the Civil War Governors of Kentucky project at the Kentucky Historical Society

Susan Perdue: Founding director of Documents Compass, Director of Discovery Virginia, and co-author of A Guide to Documentary Editing

Jennifer Stertzer: Director of the University of Virginia Center for Digital Editing, Senior Editor at the Papers of George Washington, and current President of the Association for Documentary Editing.

Matthew Pinsker: Director of the House Divided Project at Dickinson College and the author of two books on Lincoln and the Civil War era.

We gave this group full access to PAL's digital archives and to our staff. We requested their frank assessment of where the project was succeeding, where it was failing, and what could be done to produce a quality product in a timely fashion. That group identified a wide range of problems stemming from poor management and ineffective policies and procedures. However, the review team also found a talented, committed staff, as well as a collection of digital documents that will be vital to the project’s success. Summaries of their findings and key comments from their reports are below. Their complete reports accompany this summary. A more detailed plan for rebuilding the program will be coming soon. For now, the focus should be on the fine work of the Review and Planning Team, whose members took on this task because they believe in the importance of The Papers of Abraham Lincoln.

Findings First, the RAPT members were alarmed by the absence of a strategic plan. An initial vision was established in 2000, but there is no step-by-step plan for accomplishing PAL’s goals.

Feller, p. 3: “Without a change in course, what is in store is imminent and complete disaster.”

Lewis, p. 1: “PAL is in an interesting position of not having reliable data on work done and, therefore, not being able to accurately estimate the work remaining.”

Perdue, p. 5: “PAL’s lack of a clear and well understood strategic plan has hampered the project and it impacts every phase of the operation.”

Second, the team concluded the project’s scope was too large, and that this “mission creep” had crippled PAL. For example, Lincoln served eight years in the Illinois legislature. Instead of including only those bills he wrote, PAL management determined that all legislation considered by the entire Illinois House during those eight years was in scope — whether it passed or failed, whether Lincoln voted on it or not. All drafts of legislation and every document received by a committee on which Lincoln served were also included. That meant 7,344 documents needed to be located and scanned, transcribed, edited, and annotated.

Feller, p. 2: “Indeed, by ordinary definition most of the items the project has transcribed and annotated so far are not Lincoln documents at all.”

Feller, p. 3: The project’s scope “is, for all practical purposes, infinite.”

Pinsker, p. 6: “It seems to be truly paralyzing as a scope of research and utterly disconnected to the real needs of its core academic/research audience.”

Third, the review team noted that the staff faced impossible demands and unrealistic procedures. PAL staff members were isolated from the host institution at ALPLM, from their UIS employer, and from the wider field of documentary editing. Mismanagement manifested in several ways for the team. One example is that far too little attention was paid to producing a reliable transcription, as previous management eliminated a vital tandem proofreading stage. Instead, management focused an inordinate amount of attention on markup, indexing, and annotation. Another example is that the former management made no effort to get the rights to use document images from private institutions and individuals. PAL leaders never prepared a form for private owners or archives to sign granting permission to publish images. Now, PAL is faced with getting rights to more than 18,000 images from nearly 500 individuals and institutions.

Pinsker, p. 7: “I was somewhat shocked to discover that this basic step [obtaining image rights] in the research process had not been undertaken in any systematic way.”

Feller, p. 7: “It was plain that they [staff] were frustrated, demoralized, and even traumatized by the impossible demands put upon them by recent project management.”

Perdue, p. 6: “In my thirty years of work, I have never witnessed a group of people so shell-shocked, so unsure of their own abilities to do the work, so unable to figure out their priorities, and so uncertain of the future.”

Fourth, the team concluded that PAL’s relationship with the University of Illinois at Springfield created complications and led to confusion for all parties.

Lewis, p. 9: “No party – neither institution nor the employees – seems to like the current setup.”

Perdue, p. 7: “The project’s affiliation with the University of Illinois - Springfield is very problematic. … To my mind it does not seem to be necessary for the work of the project.”

Stertzer, p. 2: “This relationship needs to be discussed and either defined or

abandoned.”

Fifth, the team was distressed to find that PAL has not begun releasing the annotated Lincoln documents it was established to produce, despite having been in business for nearly 17 years.

Feller, p. 7: “It is important that the project actually produce something soon. Despite much labor, it has no product to show its public constituencies …”

Lewis, p. 4: “This project needs to show the world what it will contribute …”

Perdue, p. 11: “At this point, it is critical that the project take the conclusions from the strategic planning process and mold that into a publication plan.”

Stertzer, p. 5: “… the current state of the project – lots of work but no publication of materials.”

Finally, the review noted the dedicated and hardworking staff. More than 106,000 Lincoln documents have been collected digitally, providing a foundation for the work of getting them out to the public. The need for a complete, modern collection of Lincoln documents is greater than ever, and the scholarly world supports PAL’s goals.

Feller, p. 7: “I was tremendously impressed with the onsite staff I met in Springfield.”

Lewis, p. 7: “They’re research rock stars.”

Lewis, p. 1: “When completed, PAL will be a landmark digital publication …”

Perdue, p. 16: “There is so much good content here that needs to get out to the wider world.”

OUR PLAN OF ACTION A RAPT member summed up the situation bluntly by quoting Abraham Lincoln’s direction to a dithering general: “You must act.” The ALPLM is taking actions to put The Papers of Abraham Lincoln back on the right path. We have made several difficult yet necessary decisions including restructuring the project’s management team to better align PAL for success and moving forward without the University of Illinois at Springfield for this project. For the rest of 2017, we are dedicated to defining PAL’s scope, implementing further changes to move this project forward and providing a final project that President Lincoln and the people deserve.

Report by Daniel Feller

Member, The Papers of Abraham Lincoln Review and Planning Team

April 1, 2017

I. Introduction

I submit this report as a member of The Papers of Abraham Lincoln Review and Planning Team.

I was asked to join the team around the beginning of the year by Alan Lowe, Executive Director

of the Abraham Lincoln Presidential Library and Museum. At the time I knew of the project

only by reputation, though I have read pretty extensively in Lincoln. I visited the project in

Springfield on February 23 and 24, 2017. There I met with Director Lowe and acting project

director Samuel Wheeler. I also met extensively, partly in Dr. Wheeler’s company and partly

not, with present and former onsite staffers Kelly Clausing, Daniel Worthington, Gayle Gatons,

and Christian McWhirter. Washington staff Ed Bradley and David Gerleman joined us by skype

for part of this time. Before and after the visit I was furnished with written materials including a

special orientation packet, in-house manuals and guidelines, and the recent successful grant

application to the NHPRC. I was also given access to the project’s Pubman document database.

These, I think, were sufficient to give me a sound sense of where the project stands and where it

should go from here.

II. Current Project Policy and Progress

The first sentence of the project’s recent NHPRC application states its aim succinctly: “locating,

imaging, transcribing, annotating, and publishing documents written by or to Abraham Lincoln

during his entire lifetime.” This mission is clear and of unquestionable importance. The

project’s central difficulty lies in how far it has strayed from it. To understand that straying and

head off its recurrence, its causes may be worth considering.

Abraham Lincoln occupies a unique place in our history and our historical consciousness. He is,

I think without question, not only our greatest president (as scholarly polls regularly affirm) but

the single most important American ever. He carries an immense load of symbolic freight. He is

the common man apotheosized, the American Moses, our national saint. He inspires not only

unflagging curiosity, but a special kind of reverence. His memorial in Washington is not just a

monument, but a shrine. The mystic chords of memory and myth enshroud him.

Because of Lincoln’s stature, and because so much of his importance lies in his words—things

he wrote and said, rather than actions he performed—his documentary record attracts

extraordinary interest. Looking to explain his sudden rise from obscurity to the presidency,

scholars burrow through the scanty record of his early years for hints of the great man to come.

They pore over his literary influences and early writings for clues to the extraordinary gift for

language that he would later display in the Gettysburg Address and Second Inaugural. This

quest for the buried prize, the secret key, the hidden code that will resolve the mystery of

Lincoln’s transcendent greatness can come to look less like normal historical scholarship than

like something out of The Da Vinci Code or National Treasure. It is this that gives the Anne

Rutledge legend such power. Lincoln artifacts are treasured like relics of the saints, and even his

autograph carries talismanic power. It speaks volumes that “lincolniana” is an actual word in our

dictionary.

I am not charging this attitude of cultish devotion on the Lincoln Papers (though I observe some

of it on page 6 of the April-June 2015 Lincoln Editor, attached to the recent NHPRC

application). But I think it helps to explain how the project was allowed to slide into widening

its horizons to an extent that otherwise would be clearly recognized as unreasonable. Every

documentary edition, whether in print or online, has to set boundaries of space and time to its

work. The record of any major historical figure is too vast to present and explain in full detail.

Therefore judgments of feasibility and of relative significance must be made. The goal is to get

done someday, and to reach it one must be ready at times to say “enough.”

But with Lincoln—and with Lincoln alone—it seems that enough is never enough. With

Lincoln, we can admit no possibility of overkill. Nothing is too trivial or mundane or repetitive

to collect and preserve. We think we cannot stop until we have encompassed it all, for only then

will the supreme majesty and mystery of Lincoln stand fully revealed.

A fortifying impulse to project overreach came from within its own history. The Lincoln Papers

began as a project to document Lincoln’s legal career by amassing complete records of every

recorded case in which he was involved. For the “Lincoln Legals,” this broad definition of

corpus made some sense. Arguably one would need to see the entire record of a case, from

beginning to end, in order to fully understand the purport of, say, an appellate brief that Lincoln

filed in it. And the record of Lincoln’s law practice, while huge, at least had finite boundaries. It

was possible to do it all. One could foresee an end and reach it.

That would also be true of a project to collect and publish “documents written by or to Abraham

Lincoln.” But, whatever its origin, the project’s actual practice has belied this stated objective.

Its working policy has been to include literally thousands of documents that were not written by

or to Lincoln, but were merely associated with him, often only tangentially. Indeed, by ordinary

definition most of the items the project has transcribed and annotated so far are not Lincoln

documents at all. They are instead documents of the world around him. And the project’s policy

has been not only to include them all, but to try to explain—again, literally—everything about

them. Thus the project’s true scope has been not the papers of Abraham Lincoln, but a fully

comprehensive history of Lincoln’s times.

The project for years has been mired in the 1830s, and a look at its method reveals why. For the

period up through the end of Lincoln’s service in the Illinois legislature in March 1841, Roy

Basler’s Collected Works printed 270 documents, covering 252 pages,. Basler cast a pretty wide

net, including not only the handful of manuscript letters he located from these years, but also

routine legal documents, public communications, and Lincoln’s legislative actions, many of them

taken from printed originals in newspapers and official records.

For this same period the Lincoln project has located and transcribed 5,192 documents, of which

3,154 (61%) have been annotated and 173 (3%) have passed through the presently undefined

stage of final review.

What accounts for this nearly twentyfold expansion of Lincoln documents for these years? Some

new letters have perhaps been found, and the project, unlike Basler, counts letters to Lincoln as

part of its corpus. But the major cause is the redefinition of Lincoln’s papers to include not only

the record of his own actions in the legislature, but of nearly everything that happened in it

during his term of service. According to the Orientation Materials furnished to us, the definition

of a “Lincoln document” includes “all published acts passed by the 9th-12th Illinois General

Assemblies . . . ; also, all extant bills considered by the chamber of which Lincoln was a member

during that time; also any resolutions and/or reports introduced by or directed to a committee of

which Lincoln was a member.”

This at once transforms the Papers of Abraham Lincoln into a documentary history of eight years

of the Illinois legislature. And not only are all these things to be considered Lincoln materials,

but everything about them is to be explained in annotation. As an example of its practice, the

project’s NHPRC application offered “An Act for the Relief of Cook County,” reprinted from

The Laws of the State of Illinois. This short, straightforward, and fairly routine piece of

legislation comes with thirteen footnotes giving its complete procedural history and also a series

of linked identifications explaining such terms as “Chicago,” “Cook County,” “Illinois,” and

“Illinois General Assembly.” These in turn have links to identifications for, among others,

“Indiana,” “Washington, D.C.,” “Springfield,” “Congress,” “Great Lakes,” and “Mississippi

River.” And just in case the reader doubts the editors’ word on where or what these places are,

the notes also cite sources, such as a three-volume history of Chicago published in 1884.

This is absurd. It would be absurd even if this Act was an actual Lincoln document, but there is

nothing to show that is. Lincoln apparently did not introduce this law, or speak on it. It is not

even clear that he voted on it. He just happened to be a member of the legislature that passed it.

The project newsletter Lincoln Editor (enclosed in its NHPRC application) offered this sample as

“a tantalizing glimpse of what is in store.” Indeed it is, in the same sense that the Titanic’s first

sighting of the iceberg offered a tantalizing glimpse of what was in store. Because, without a

change in course, what is in store is imminent and complete disaster. Documenting Lincoln’s

legislative years in this exhaustive manner has all but paralyzed the project’s progress in the

1830s. Doing it for his two years in Congress in the 1840s will do the same again. We will

never get to the presidency.

That is perhaps just as well, for if the project’s defined scope for the pre-presidential years is

impossibly large, for the presidency it is, for all practical purposes, infinite. The project intends

to consider as a Lincoln document, and to annotate in the full manner above, “all acts passed by

Congress during Lincoln’s presidency,” every order issued pursuant to his authority whether

under his personal hand or not (which means pretty much every directive issued by his

administration), and “the entirety of any document enclosed within correspondence that is

addressed to Abraham Lincoln or written by Lincoln.”

During my visit, I asked the staff whether this policy meant that every enclosure within a Lincoln

message to Congress would be considered a Lincoln document, and would therefore receive the

kind of full annotation accorded to the Illinois legislative materials, including identifying every

named person, place, organization, and event. The answer was a quick “yes.”

Here’s where this leads. On December 8, 1863, Lincoln communicated his third Annual

Message to Congress. The message itself is only a few pages long, but, accompanied by reports

of the various executive departments—all of them addressed to Lincoln, and in turn enclosed by

him to Congress, and therefore all Lincoln documents by project definition—it takes up six

volumes of the congressional Serial Set. It comprises, in effect, a full and detailed history—

diplomatic, military, naval, financial, administrative—of a year on the Union side of the Civil

War. Fully annotating this one message, by itself, would be at least a decade’s work for a single

editor.

III. Recommendations

My simple and central recommendation is that the project return and strictly hew to its stated aim

of “locating, imaging, transcribing, annotating, and publishing documents written by or to

Abraham Lincoln.” Without close adherence to this mission, I see no prospect of resuming

progress toward completion. Even reaching the presidency under the present plan will never

happen.

Here are some more specific recommendations:

1. Scope and definition

The project’s present conception of Lincoln’s corpus is not only unrealistic but nearly fetishistic.

It embraces thousands—and in the presidential years, probably tens of thousands—of documents

that no one could claim tell us anything significant about Lincoln. It is a policy explicable only

on the nearly magical idea that somehow the dashed-off “A. Lincoln” on a document gives it

talismanic power.

The project should return to a common-sense definition of a Lincoln document—not every one

to which he can be connected, but only those written by or to him and substantively involving

him. Even in the 19th century, presidencies generated a large mass of paperwork, much of it

utterly routine and revealing essentially nothing of the president’s personal imprint even when

bearing his signature. Other presidential papers projects have developed policies for separating

the wheat from the chaff in this mass, and their leads should be consulted. For Lincoln’s

congressional as well as presidential years, the definition of a Lincoln document should be drawn

to avoid pointless duplication of the U.S. House Journal, Congressional Globe, and Statutes at

Large, which are available in free searchable format from the Library of Congress. Also

excluded should be all document enclosures printed in the congressional Serial Set. I would also

recommend excluding the bulk of the Illinois legislative materials, were it not that the work on

them is apparently already largely done.

The following, which (according to the orientation packet) by present policy are specifically

included as Lincoln documents, ought in my view to be explicitly excluded:

A. Second-level enclosures, such as executive department reports enclosed to Lincoln by his

Cabinet officers for inclusion with presidential messages. Many of these were officially printed

in the Serial Set or elsewhere.

B. All congressional acts and proceedings during Lincoln’s House term except those reporting a

specific speech or action (not just a recorded presence or vote) by Lincoln himself.

C. Acts of Congress during Lincoln’s presidency. The only thing that makes these Lincoln

documents is his signature. They are already available, conveniently lined up, in the Statutes at

Large. Including them without annotation adds nothing to our knowledge, and annotating them

turns the Lincoln Papers into a history of the Civil War Congress, which must urgently be

avoided.

D. “General orders, orders, proclamations, directives, and other documents, whether handwritten

or printed, emanating from cabinet secretaries at the request of President Lincoln or on his

behalf”; also documents issued by Lincoln’s secretaries “at the request of President Lincoln or on

his behalf” [quoted from the “Scope” section of the Orientation Materials]. Including these

would stretch the project’s boundaries almost without limit.

E. All routine official documents on which Lincoln’s only contribution is his signature. Among

these, from the “Document Types” listing in the Orientation Materials, are appointments,

commissions, and nominations (unless handwritten); exequaturs and other credentialing

materials; passes; and treaties. A general rule should be to exclude all printed blanks or forms,

as well as standardized documents such as proclamations of land sales, unless Lincoln wrote

something particular on them. Including all these repetitive documents tells us nothing about

Lincoln except that he signed them, which we already knew.

2. Annotation

The annotation policy laid out in the fifth paragraph under Workflow Stage: Annotation:

Contextualization on page 8 of the Orientation Materials (beginning with “Annotation should not

provide . . .”) is sound and wise, but has evidently been disregarded. Actual current practice is to

identify and explain everything specific in a document, no matter how obvious or unimportant.

The aim seems to be to make the Lincoln Papers a self-contained, standalone entity, a

comprehensive reference work in which readers who know nothing about Lincoln, not even the

name of the country he lived in, can find the answer to every conceivable question. This policy

should be repudiated.

Mary-Jo Kline’s standard Guide to Documentary Editing offers a good discussion of best

practices in annotation. Its general message is that annotation should hew closely to the

document at hand, and confine itself to explaining only what is necessary to make sense of it.

These are usually things that the writer and recipient know but the reader does not. The Guide

concludes: “When the editor suspects that a footnote is unnecessary, it should be omitted. In

imposing annotation on a documentary text, the best rule is ‘When in doubt, leave it out.’”

Under this prudent rule, nearly every note to the sample “Act for the Relief of Cook County”

discussed above should be stricken (even if the Act itself is considered a Lincoln document,

which it is not). One can fairly debate how much prior knowledge editors should assume on the

part of the reader, but they must assume some. The project now presumes none — that readers

will not know what Illinois or Congress or the Mississippi River is, and therefore must be told.

In fact they will know; and if they do not know, this is not the place to tell them. Such

information is readily available elsewhere. The Lincoln edition will not exist in a vacuum.

Indeed, it is highly unlikely that readers lacking certain basic knowledge will even find their way

to it.

So how far should annotation go? The project’s NHPRC application chides Basler’s Collected

Works for inadequate annotation, but in practice I have found most of Basler’s notes to be

entirely sufficient. Other recent editions of 19th-century statesmen’s papers also offer useful

models.

Most of the specific things, and types of things, prescribed for annotation on pages 9 to 11 of the

Orientation Materials should not be annotated. We do not need biographies of every

organization named in the documents; in the cited example on page 9 of “a republican club in

New York,” its very name tells readers most of what they need to know. Notes in the “Events”

category should be narrowly circumscribed, and the “Places” category probably ought to be

stricken altogether, with individual places noted only where something particular about them is

integral to understanding their mention in a document. Identifications of people should be held,

in most cases, to one or two sentences, and should likewise be constricted to information

pertinent to their appearance in the document, avoiding full biography. It is customary in

editions, when first introducing persons, to give birth and date years. Further detail of exact birth

and death days and localities is superfluous; leave it out! This will also save much staff time and

trouble, since such information is often hard to find, unreliable, or contradictory.

It is apparently current policy to require citation to external authority for every fact stated in the

notes—to let no statement stand unsourced. This should be changed. Much of the information

given in the notes will be uncontested and in the public domain. Where it is not, the editors

should be trusted to be certain of the correctness of what they say. If they are not certain, they

should say nothing: better to omit than to err. By the time they sit down to write annotation, the

Lincoln editors will know their subject better than anyone else in the world. It is simply perverse

to stifle their expertise by turning them into voiceless pass-throughs, barred from presenting any

fact that they cannot attribute to someone else. Many statements in annotation will not be

traceable to single sources in any case. The only sources that are proper to cite in annotation are

contemporary materials. There should be no mention of (or reliance on) modern secondary

writings. Excluding them will protect the edition against premature obsolescence. It will also

beneficially rein in the impulse to wander into discursive or interpretive annotation.

As examples of present practice, the staff told me that terms like “flintlock” and “saline reserve”

presently receive footnotes giving definition and citing sources. “Flintlock” needs no footnote,

unless it appears in a document discussing something very specific to it, such as comparing its

reliability to that of a caplock. “Saline reserve” may need a short explanation, but in neither case

is there any reason to cite a source.

In all this, again, Basler and certain modern editions can furnish useful models. If this policy

seems too constrained, consider that thousands of people have read and comprehended the

Lincoln-Douglas debates or various Lincoln compendiums, including the excellent two-volume

Library of America Lincoln, in editions which have either very few footnotes or even none at all.

Annotation is ancillary. It should facilitate, not suffocate. Let the document itself remain the

primary focus; and let it as much as possible explain itself.

3. Presentation

I will leave advice on digital content management systems and publishing platforms to team

members with technical expertise in those areas. However, I suggest fundamentally rethinking

the present plan to mount high-resolution digital color images of every Lincoln document found.

First, it is expensive and physically difficult. Two staffers are now employed full-time searching

and scanning in Washington, and gargantuan tasks of mounting and maintaining loom ominously

beyond that. The Washington staff described to me the obstacles encountered in scanning

documents from the National Archives. Some of these items have already been microfilmed but

are being re-scanned from the originals to obtain better images—or, as I was told, to include

blank pages that were left off the microfilms. I see no point in this.

Another problem is that mounting images of privately held documents requires permission of the

owners, which has not always been obtained. The staff is rightfully dreading the time and effort

it will take to go back and secure it now.

And all this to what end? It is not intrinsic to an edition’s mission that it present images of

originals. Indeed, the very point of an edition is to supersede those by giving us something better

in their place—to make viewing originals unnecessary by presenting faithfully transcribed

versions that are vastly easier to read and utilize. Until recently, disseminating original images

in mass quantity was simply out of the question because the wherewithal to do it did not exist.

But the fact that it is now technologically possible to do it—or at least to conceive of doing it—

does not of itself make it worth doing.

Some projects, including my own, have linked our online editions to original images where those

are already readily available, for instance in a Library of Congress digital collection. This

linking involves only minimal time and expense. I suggest the same strategy for the Lincoln

project. Jettison the plan to mount new images of everything on a new project platform. Instead,

use the site already established at http://papersofabrahamlincoln.dataformat.com for displaying

National Archives and Library of Congress document images. As the edition is published, link

its annotated transcriptions to the images posted there. For documents obtained from other

sources, continue securing facsimiles for in-house use. But forget mounting them online. This

plan, together with returning to a sensible standard for defining a Lincoln document, will obviate

the need to seek permissions, slash the amount of labor ahead in Washington, greatly reduce

expense, and speed the project toward completion.

4. Staffing and office management

I was tremendously impressed with the onsite staff I met in Springfield. The project needs to

find a way to better deploy their skill and expertise and to restore their morale (and raise their

salaries, which are depressingly low). Kelly, Daniel, and Christian have a deep knowledge of

their materials and an accurate sense of what should be done with them. It became clear in our

conversations that the futility of the past plan of operations is as obvious to them as it is to me.

They were very discreet in their remarks; there was no disgruntled airing of grievances or

mention of names. Still it was plain that they were frustrated, demoralized, and even traumatized

by the impossible demands put upon them by recent project management. They understand the

difference between the papers of Abraham Lincoln and an interminable history of the world.

They want to move forward, and to that end they want to return to sensible policies of scope and

annotation. They want to work on actual Lincoln documents, to produce pertinent and genuinely

useful annotation, and then to move on.

Since these people’s invaluable expertise is key to the project’s success, their labors should be

carefully guarded from diversion. According to its NHPRC application (and confirmed in our

discussions), the project “answers scores of questions by e-mail and phone each year.” This is

good to the extent that it helps bolster project visibility. It is bad to the extent that it steers

valuable staff time and expertise into private and often trivial channels. I suggest distinguishing

between public and private queries. Public queries—from government officials, news agencies,

and the like—should be fielded within reason, and the inquirer should be urged to credit the

project publicly with the information furnished. Private inquirers, such as local historians and

genealogists, should be intercepted and dealt with by museum personnel, volunteers, or student

assistants. Professional staff time should be reserved as much as possible for the project’s core

mission—to produce a Lincoln edition, not to provide a free information service or reference

library.

5. Publishing Series II

It is important that the project actually produce something soon. Despite much labor, it has no

product to show its public constituencies since the Lincoln Legals (now Series I) in 2009. The

1830s materials should be wrapped up as quickly as they can be, and editorial work from there

on should be ordered chronologically as much as possible. That way, cohorts of documents,

each covering a year in Lincoln’s life, can be completed in order and (once a platform is

established) released in sequence, just like volumes in a traditional edition. This will bolster

momentum, as the staff feels itself making progress through Lincoln’s life, and also provide a

public show of activity.

It appears that transcription of documents up to 1841 is complete, and that within that period

legislative materials are so preponderate in number that other Lincoln documents—what I would

call real Lincoln documents, such as letters—are all but lost among them. For that reason, the

project may wish to consider separating out the legislative materials and publishing them as a

separate sub-series. This presumably could be done quickly, and again would afford evidence of

forward progress.

Narrowing he project’s scope and annotation policy, which I think is urgently necessary, raises

the issue of consistency in presentation. Now that some documents have been excessively

annotated, and many documents have been included in the corpus that probably should not have

been, does consistency require either continuing on the same plan from now on, or alternatively

going back and redoing what’s been done to align with the new policy going forward? If the

work was just beginning, I would say yes to one or the other. But at this point, consistency be

damned. Too much work has already been put into the early documents, and the main object

should be to avoid putting in more. Wrap them up and move on. Having different documents

annotated to different standards may compromise the edition’s uniformity, but will not undercut

its quality. No user will be hampered by it, and most will not even notice. Likewise with the

redefinition of scope, and the exclusion in future of tangential documents like legislative

materials. Do not worry about the critic who, years from now, will say, “you counted Illinois

legislative bills as Lincoln documents, so why not Acts of Congress?” That critic does not

exist—and if he does, should be ignored. The aim of the project is not to achieve perfect

comprehensiveness. It is to publish documents written by and to Abraham Lincoln—and, within

the foreseeable future, to get done.

Daniel Feller

Professor of History

Distinguished Professor in the Humanities

Editor/Director, The Papers of Andrew Jackson

University of Tennessee

1

Papers of Abraham Lincoln Review and Planning Team Report

Patrick A. Lewis, Civil War Governors of Kentucky

The leadership of the ALPLM and of PAL have done a very bold thing in assembling the

Review and Planning Team, and that should be heartily commended by everyone in the fields of

scholarly editing, public history, and U.S. history. It is a testament to the commitment of this

institution to the project that so much energy and effort is being expended to make sure this project

reset is carefully planned out. When completed, PAL will be a landmark digital publication not

only for scholars but for editors. Its influence is already seen in the DNA of the project I direct,

and I expect that I won’t be the last to make that claim. This project had a vision—an inspirational

and forward thinking one. I hope these comments help the project find that vision again, and find

new ways of working that will carry that into reality.

Editorial Work

At this moment of reassessment, PAL is in an interesting position of not having reliable

data on work completed and, therefore, not being able to accurately estimate the work remaining

to do. I start here because regardless of the scope and depth refinements that need to be made, there

is no reliable data on how this will actually affect the completion date. The project needs to take a

serious look at its own history and measure its progress in different ways than it previously has. I

get the sense, too, that only the previous project management had access to data and metrics (short

of individuals knowing their quota of work to do in a year). To prove good faith to federal and

private funders, PAL should recreate itself as a data-driven project, in which staff and leadership

work together to own progress towards completion and to understand realistically what that will

entail.

Again, this requires different data—data which I believe is recoverable from current

statistics, but which will require reconstruction from deep within the project archives. The DC

search is high on the priority list here. Combining the number of documents accessioned with the

number scanned per year tells project leadership little. Those need to be two separate metrics,

particularly given that the scanning work is often taken out of the hands of PAL staff by NARA

policy. A more useful metric would be to divide the number of cubic feet processed thus far by the

labor hours invested. The number of documents found and the scanning rate can come and go as

they may, but PAL has to have a handle on the scope of the work ahead of them.

2

The statistics on editorial work are likewise muddy. Partly, this is due to the previous

management which changed policy seemingly on a whim and gave editorial staff significant clean-

up work to bring documents up to current standards. Consequently, the staff seems not to clearly

understand the division between editorial processes and how long each of those steps should take.

Staff need to own editorial policy, to be included in its development, to make firm decisions after

discussion, and to stick to those decisions for the sake of the integrity of the edition and for accurate

projection of the future work left to be done. Just like the funders, the editorial staff are

stakeholders in this project and deserve to understand how each of their daily hours contributes to

the project as a whole. With bad metrics and a repetitive cycle of revision, they’re drowning in

endless work.

The editorial policy documents need to be rewritten from the ground up. The editorial staff

should work together to decide what needs to exist in the final edition and then needs to create a

new work process that distributes that work evenly across each stage. Then, in contrast to previous

practice, the project leadership and staff must hold one another mutually accountable for standing

by those decisions. If transcription is transcription and nothing more, so be it. If you don’t tag X

element at this stage, don’t tag it. The staff should have a sense of what is too much work at one

stage. And as the staff develops new policy, they need to test run it on some dummy documents

and reevaluate before making a final decision. Evaluation and deliberation is not time lost

compared to fighting flawed policy for decades.

In the reevaluation of editorial processes, PAL needs to think clearly about the role of

documentary editors. The impulse should shift from doing scholars’ work for them to allowing

them to do the work they want to do. Make their project possible, but don’t make it your project,

too. This most is apparent in the annotation stages, in the lengthy footnotes on the legislative

history of a bill or in the contextualizing work on the saline lands. It is also critical to evaluate this

in XML coding. There is too much code. Tagging hand shifts is tough to justify when you also tag

to identify that individual. That’s double work and far more clicks than there need to be. God

forbid this happen in a petition. And this is only the most prominent example. How critical is

marking up each element of a letter? Is the time invested worth the actual use by a researcher?

Will that change the way a document is displayed? Will that display be useful or intuitive?

What is more concerning is that this code is being applied to an unverified text. It is time

consuming, but PAL needs to add an oral double proofing stage. Two editors need to sit down and

3

read that MS aloud and resolve all questions about the transcription before much code is applied

to it. No matter how many individual humans look at a text, it is in our DNA to unconsciously

resolve minor errors. This is particularly the case when the eye sees text that is cluttered, in latter

editorial stages, with meaningless XML. Two independent brains are the only way to beat

ourselves. The text comes first. Then do enough to present the text in an intellectually defensible

manner. Then do enough to enable users to find the text. That is all.

There is a big question about the scope and aims of the project. This affects more than just

what PAL publishes, it affects how it presents and annotates its texts, so I’ll address that now. Is

this project about Abraham Lincoln, or is it about the society in which Lincoln lived? Both are

valid and valuable projects. If PAL is a biographical edition, then editorial processes from selection

to tagging (red-letter tag Lincoln’s words?) to annotation orient themselves to biography. If PAL

is a societal edition, then Lincoln may well get lost sometimes, and that’s OK, too. CWGK, for

example, rejects the biographical model and has chosen to be a project about the society whose

lives are recoverable through the archival privilege of the 5 wartime governors of Kentucky. That

works for us; it might not for PAL. PAL was a great advisor for CWGK early in its project history

because, I can now see, the tendency of the previous project management had been towards a

documentary history of the society in which Lincoln lived and worked. That is a much larger

animal to wrangle, given Lincoln’s broad archival footprint—which is not to outright discourage

PAL & ALPLM from considering tackling it. But the project needs to clearly articulate its goals,

because decisions about inclusion, depth of annotation, level of textual encoding, funding, and the

need for and shape of partnerships all stem from that philosophy statement.

I strongly suggest a staff retreat to a conference room or a state park for a few days with

the goal of crafting a few sentences of strategic vision in which every word counts. And, just like

the editorial policies that flow from that vision statement, everyone invests in them and lives by

them. Each of CWGK’s editorial stage policy documents has a mission statement at its head. As

we build that document or when we revise it, we have to make the case to ourselves that every

element lives up to that mission and achieves the outcomes we want from the stage. If that work

fits better philosophically in some other step, it goes there. If that proposed idea doesn’t fit with

where we see the project going as a whole, we don’t do it. You’re re-writing your project

constitution. The Articles of Confederation let PAL muddle through in good times and collapse in

crisis. We need to build a new governing platform.

4

And much like a sound, Constitutional government, PAL needs to get used to saying no to

itself. Doing so (after thoughtful, respectful discussion between staff and leadership) does not cut

down on the usability or integrity of the edition. Indeed, it enhances integrity by checking the

tendency to change course midstream. And it enhances usability because it will ensure the timely

delivery of a product.

Publications

Does PAL and ALPLM want an immediate publication? Do the funders (federal, state, and

private) deserve or demand some sign of good faith effort? If a publication is deemed desirable in

the next couple of years, will that be the first piece of a permanent home for PAL, or a waystation?

In Lincoln terms, are you building a cabin that might turn into a dog-trot that might turn into a

framed house when the family grows and time and budget allow? Or are you building a lean-to to

get you through a blizzard but you’re not ready to settle down yet.

My inclination is to believe that some sort of proof-of-concept publication is desirable.

That should take one of two forms: 1) a large-ish number of documents (few thousand?) with

incredibly light markup and annotation (leave your coding in, but don’t have the style sheet display

it), or 2) a small number of documents (one or two hundred) done up not to the level of

“completion” according to current standards, but that look like the staff and leadership want every

document to look. And that publication site must clearly explain what is and is not there, the

rationale behind the publication, and the questions that PAL wants to answer in the publications

process. Either way will be a valuable learning experience for the project.

Contrary to what CWGK elected to do with an Early Access site, I would suggest option

2, a “full” work-up of a handful of items. In truth, digital Lincoln is a crowded market. People

have access to unannotated Lincoln with Basler, LOC, the digitized OR, House Divided, and more

available. Yes, PAL has some interesting early Illinois material in the hopper, but I think it’s more

important at this stage to show (and to craft before you show) the intellectual vision of the project

than it is to deliver new content. This project needs to show the world what it will contribute to

rebuild investment, emotional and financial. This will also allow the staff to field test the new

editorial policies they will be developing. It might even be wise to strip your selections down to

raw transcriptions and work them up anew.

5

Now, what that looks like from a tech perspective is up to PAL. Drupal seems the likeliest

candidate. I would suggest hiring a full-time digital humanist who can handle the heavy lifting of

development and design. You can contract out holes in that person’s skills, but the underlying data

and project management infrastructure with PubMan is strong enough that there won’t have to be

much building—mostly interface design and planning search function. Again, how permanent this

proof of concept publication will be is up to PAL. There is a different set of strategic concerns

involved in showing the intellectual access and apparatus you will provide versus showing off

content. With a DH person on staff, though, they can probably start to make your lean-to into a

home if it’s thought out well from the outset. Again, though, this is why a smaller but more

technically complex initial rollout is desirable. It allows PAL to think through the big issues, tinker

with the margins, and have something to fundraise out of. I feel like this gives the project more

marketability than a big dump of raw texts. It’s your vision statement to match that new

constitution. That informs the pitch to private funders and to the feds.

Whatever that initial rollout looks like, it has to fit into a strategic publishing vision. This

needs to prioritize core documents for full workup, clearly break up workflow into manageable

chunks, needs to have a solid grasp of statistics to understand realistic annual production, and think

about how PAL markets what it produces—and to whom.

ALPLM and PAL need to take stock of what is already online: Chronicling Illinois, the

NARA images database, and LPAL. Fixing LPAL should be top priority and doesn’t sound too

difficult (knock on wood). I would resist the temptation to bring that “officially” into the scope of

PAL (certainly now, possibly forever), but fixing it and promoting it—and promoting the fact that

you fixed it—would be a definite sign of progress and a signal to everyone watching that the ship

is starting to turn. But that’s probably all. Lots of the scope creep began from LPAL seeds; PAL

needs to close the book on that for the time being and focus on its own identity.

Chronicling Illinois? From the limited conversations we had about that site, it sounds like

there are some significant ALPLM questions to ask about this platform, its administrative history,

and its strategic purpose. Maybe it has little to do with PAL. Maybe, though, it could be an outlet

for some of the Illinois-specific material that is interesting, is unpublished, but isn’t core Lincoln.

Maybe, given that Omeka is an easy exhibitions development platform, it’s PAL’s quick and easy

way to curate online lessons and curriculum materials. Maybe it is a tool to help rebuild those in-

the-building relationships with the ALPLM staff of other departments, finding some sort of

6

collaborative activity to work on their materials. Regardless, it needs to be considered in the

strategic publications vision of both PAL and ALPLM. Dead-end sites are bad investments if you

continue to pay the server fees for them but never integrate those into the institution’s current

programming and other resources.

I think the NARA images database is a huge asset for PAL. It is essentially a new

publication—I certainly didn’t know it existed—with no new costs. All you have to do is publicize

it. Take a bit of time to write up some documentation, some user instructions, and a pretty lengthy

discussion of the scope of what is extant on the site. Be forthright about the acquisitions

inconsistencies and scope creep. It’s not airing dirty laundry, it’s good editorial practice. And,

moreover, you’re doing a service to the field by demonstrating where good scholarly intentions

can lead a project astray through small incremental decisions. Publish that. Talk about it at ADE.

Make the error useful, not shameful. Doing so will also respect and validate the institutional

knowledge of the DC search team while bringing them into this healing and rejuvenation process

from which they seem largely out of touch. They need to own some of that process (both what has

happened and where the project is headed) if they are ever going to truly buy in to the relaunch.

There doesn’t seem to be much evidence of that buy in from DC right now, and that’s a very

dangerous thing for the cohesion of the staff, the new editorial vision of the project, and the

potential loss of institutional knowledge if/when those folks part ways with PAL.

Back to the NARA images site, once the larger vision for prioritizing documents is in place,

talk about what documents will be moving forward in the editorial process now, what will be

worked on later, and what you’re going to leave here at this stage. This is the perfect safety valve

for the copious court martials and post road petitions that fell into those Lincoln-may-have-looked-

at traps as the scope spun out of control. The labor of identifying and scanning that material won’t

be lost if it is available through images and metadata—it’s still online rather than in

Washington/College Park. In time, you can sell this as your appendix.

Eventually, publications should look like a pyramid. Wholly annotated and networked core

documents at the top, then peripheral series being textually marked up and maybe with some

degree of annotation, and finally large appendices of images and metadata or the most raw of

transcriptions which only the most serious researchers will ever plumb. As time and funding allow,

and driven to coincide with emerging scholarly questions or ALPLM programming and exhibitions

priorities, new document series can get slated for editorial work to move up the pyramid. Ideally,

7

a single search apparatus allows researchers to search each of these levels of publication and move

between them confident in what they are seeing.

Print publications may come into the picture, but only as promotional or educational tools.

CWGK at one point had considered a user interface that would allow educators to select a set of

documents and annotations and send that to a print-on-demand service for course readers. The

demand for that service (of which there is none) has not justified the cost in building that interface

(which would be substantial). Having something tangible (as PAL has done in the past) isn’t a

terrible idea, but it can as easily be a color brochure as it can be a book—far easier and less

expensive, in fact. But your real selling tool should be the proof-of-concept interface and sample

documents that go up in this first publication. That site live on a tablet or projected in front of a

meeting sells your concept better than anything on paper ever will. The federal funders don’t

demand print, it conveys no extra legitimacy, and distracts born digital projects from the important

work of public engagement that they are uniquely situated to engage in.

There is one other publications item that needs to be considered: the over-annotated

secondary literature that the editorial staff has produced. The saline lands business and any other

similar examples fall into this category. It has to go. That sort of excessive footnote is out of place

in any edition, and compared to what will be produced in a leaner editorial policy, it will be all out

of balance. But you don’t want to throw away that effort. That is valuable reference and

interpretative material and should see the light of day, but PAL needs to carefully consider how it

wants that material to support the mission of the project. Is it a digital reading room adjacent to

the main database? Is it a series of staff-authored articles in relevant journals? Is it dripped out in

a series of blog posts that highlights the specialized knowledge of PAL staff? (In any case, it should

all go to the credit of the staff who produced it—they’re research rock stars). There is something

important to be said publicly about the breadth of knowledge that editors amass—a fact that

academic researchers miss as they plunge into the archive and build depth of knowledge in a silo.

It’s a different way of knowing about the past, and this is excellent written evidence of what editors

know and also how they know. There is a professional development lesson there beyond the

content, if applied correctly.

But these are considerations of time and money, again, which need to support only the

prime goals of the project. What helps the project editorially and promotionally? When time

allows, find a way to use that data anywhere but in the footnotes to documents.

8

Sustainability

It seems that many of the problems with the previous administration of PAL was a belief

that the Lincoln name was enough to carry the project alone—without support, advice, or

meaningful partnerships. In the convening of the Review and Planning Team, the current

administration is marking a clear break in that philosophy.

CWGK has become successful because it has become a major engine of KHS exhibits,

education, outreach, publications, and public programming. We have delivered excellent content

and creative interpretation—at a real cost in staff time—to colleagues across the building. And the

institutional leadership and KHS Foundation have recognized and rewarded that effort with

opportunity and resources. The federal funders notice it, too. PAL has these same opportunities

and should build in collaboration time into future projections of workflow.

Internal relationships within ALPLM need rebuilding as well as external relationships. For

as much unique early Illinois content as the project has in hand, it seems an easy matter to start

reconstructing local relationships and rebuilding an energized local donor and user base. Back to

a KHS example, because CWGK has materials from every county in the state, we have become an

easy first stop when a colleague is going to do a talk to a school group, library, or Rotary. More

importantly, our Foundation staff know that they have meaningful local resources available to

them on their tablets as they walk into a prospect meeting. This connects the mission of what has

been seen in the past as a faraway state museum to the concerns and contexts of local stakeholders,

politicians, and donors. Such a documentary edition is a powerful goodwill tool.

PAL has roots in the statewide search for LPAL documents, and has added to that with

collections of resources concerning the entire state from Lincoln’s legislative work, added to the

state and national appeal of Lincoln on his own merits. CWGK has been able to make two

successful pitches aimed at two distinct audiences: talking historiography and digital humanities

to academics and federal funders, while talking about addressing contemporary state issues in the

laboratory of the past to local constituents. PAL has this capacity.

The board situation sounded rather complex during our discussions in Springfield. I don’t

know how bound the project may be to any current setup or membership, but finding some way to

signal a course correction through some board restructuring would not be a bad plan. Editors and

scholars should make up their own advisory board. ALPLM has to decide whether an independent

9

fundraising board for PAL is wise, or whether that should fall under the broader institution’s

umbrella board. In any event, some clear private fundraising goals need to be stated—and attached

to a realistic fundraising strategy. Fortunately, the reset gives private funders small(-ish) and

quick(-ish) achievable that immediately reward a donor base and can begin to warm up some

bigger fish for larger asks down the line. They’ll see a product quicker than in the past and—

hopefully—tell their friends. As part of the new data collection plan, too, PAL should have a better

and more accurate itemized list of outstanding needs and an understanding of when those items

need to fall into place. From that, the new leadership can strategize which pieces go into grants

and which seem appropriate to take to the foundation.

One of the largest relationship questions comes from staff appointments and the

relationship with UI-S. No party—neither institution nor the employees—seems to like the current

setup, but there may be deep politics involved in getting anything changed about the financial or

institutional relationships. If ever there was a moment to try, however, now certainly seems the

time. There may be some value in showing a university partnership in a grant application; that may

be a box that some academic reviewer wants to see ticked. But if the system is a headache

administratively and a source of worry and insecurity for the staff, it might not equal out to being

valuable. Obviously, CWGK has gotten along pretty well with state hires and without any major

contribution from a university. At the time CWGK was launched and still today, both state

government and the state university system were experiencing similar cuts, and each institution

(understandably) had to be loyal to its core identity and mission. Unless there is some national sea

change in how history departments teach and train students using the skills and resources of public

history institutions and see museums and documentary editing projects as vital classrooms to teach

practical skills to history majors, PAL will always mean more to ALPLM than it will to UI-S. The

fact that the PAL staff at UI-S is hosted in a public policy center makes any such increase in

appreciation for PAL at that institution even more unlikely. If PAL moved staff salaries into the

ALPLM side of the house—maybe finding a new way for UI-S to contribute through hosting or

student participation, or maybe not—I think it would be an incredible and valuable vote of

confidence in the remaining staff.

I don’t know the financials, but taking those positions stateside might mean they cost more

when you roll salary and benefits all together. But PAL has been known for high turnover in the

past—though, of course, much of that was induced by previous mismanagement. Showing stability

10

to potential funders and respect for commitment and institutional knowledge to remaining staff

might be well worth that. There is probably going to be a slow build-up back to the required

staffing level to realistically complete this project—and it probably won’t ever be as robust as the

staffing was years ago. PAL has to earn trust again from the inside out.

PAL reintegration into ALPLM also has to come in marketing and messaging. The project

needs to make its rebirth known and to make the persuasive case for reinvestment of trust and

resources. It will be up to ALPLM to decide whether PAL gets its own social media presence or

becomes a regular contributor to the larger institution’s accounts. CWGK’s setup is an enviable

one with a WordPress site that project staff control and publish on regularly and then pass that

content off to KHS social media for publication. KHS has a much larger footprint than CWGK

ever will. And, moreover, putting PAL into the stream of exhibits and collections posts that

ALPLM regularly runs will help reinforce this idea that the edition is a valuable (and valued)

department of the institution, serving not only the scholarly goals common to documentary editions

but supporting the broader forms of public engagement with history that can set a project apart

from its peers without a real public facing outlet.

Staffing & Leadership

So, what does the build-out look like? Director first. There was some discussion on-site

about what that position should look like. It should first and foremost be an editor—and likely a

digital one, to boot. The problems that PAL ran into stemmed from being too tempted to include

“neat” and “interesting” and never make the hard decisions that editors make. Scholars get

tempted; directors stand firm. A Lincoln background, or a 19th century background is useful, but

everyone in the running will have gone to grad school and can learn on the fly. You want a good

decision maker first—fair, open to staff commentary, but willing to make a call and stick by it.

That person should be secure in themselves and their professional identity; fear and insecurity have

done untold damage to this project in the past. That person should understand that this will be a

long overhaul and should be invested in that process. That person must be able to authentically

communicate investment in that process to a wide variety of audiences, but principally to your in-

state stakeholders and to the federal funders.

The director will have to sell this project back to the staff, back to the institution, and to

everyone outside of the building—and in many ways that will be harder than building from the

11

beginning. They will have to sell it back to the field of documentary editing, where the necessary

staff to build back out will come from. That means they need to be known around ADE and trusted

in order to attract talent and build the sorts of partnerships and advisory relationships that will be

necessary to a successful restart. Just like they are with the staff, the new director will have to be

comfortable soliciting input from advisors in the field but also must be comfortable in making a

decision that is right for PAL and sticking by it. But that takes editorial expertise and experience

to have a feeling for the limits of what is possible or beneficial for a project of this scope—and

this project more specifically.

It will be important for the director to help rebuild policy and bring the staff back together

around new objectives—to lead that Constitutional convention. Even if the primary duties of the

director are primarily outward-facing, that person must thoroughly understand the work at every

stage and be able to translate policy into XML. They should not be afraid of getting their hands

dirty with real editorial work, particularly during the first few months. They need to feel the flow

of the project and know what they’re asking of their people. The staff deserves someone who

knows these processes so they can have faith in the editorial decisions the director makes. And the

project needs the director to have that feel to accurately project workflow into the future. The

policy reassessment and the building of the initial publication site should be the top two priorities

of the new director, and that work should set the tone for the first grant applications that will go

out.

An assistant director responsible for the close supervision of editorial policy in day-to-day

work is next. An internal candidate might be suitable, or the new director can launch a national

search. Timing is critical here, though. PAL might want this person to be present when the new

policies they’ll oversee are developed and deployed for the first time. From that perspective, it is

tempting to suggest that ALPLM should release the call for both director and assistant positions at

the same time, but I hesitate to recommend that. And, truth be told, the timing of this position

might be dependent on the budget. Someone—whether it’s the director or the assistant—needs to

spend some significant time in DC or talking to the DC team and reevaluating that search

process/procedure. Aside from that work, there are staff cohesion issues between DC/Springfield

that need to be addressed. The assistant director seems the likely candidate to take charge of that

process in time.

12

I would suggest, eventually, two specialist full-time staff. First, a web developer/digital

humanist, as discussed above. If budget is an issue and with the need for a proof-of-concept

publication on the near horizon, I could see an argument for making this the first hire after the

director and before the assistant. Later—maybe years later—I would suggest an

education/interpretation specialist to tap into the existing resources at ALPLM and within IL state

government to get PAL materials in front of as many audiences as possible. At first this might not

be a standalone position—or a full-time one. It could be nothing more than a recognition that some

of the time of existing editorial staff needs to be devoted to the development of these products and

shopping them to teachers, etc. It could be a half editorial/half interpretation position (which may

be ideal to embed that person in the documents and in the office culture). But to make the most

out of being housed at a public history institution—which, again, can be a competitive advantage

for PAL in a world where the federal funders need to demonstrate relevance and public

engagement with hard numbers—this needs to eventually be staffed.

For the front-line editorial and search staff, PAL needs to get a handle on the work it plans

to do. The 2-2 core in place now is capable of carrying out the first phase or two of the rebuild.

Once PAL has an estimate of the labor hours on the plate (given the new strategic publications

plan and based in new, more reliable metrics), then the project can look to adding editors. Given

the backlog that currently exists, I don’t know that I would add any DC positions for some time to

come. Editorial work in Springfield should come first. Add as funding comes available after filling

the positions above. PAL could get creative in using grad students to chip away at the lower levels

of this editorial work. Maybe that’s the basis of a new relationship with some nearby academic

partner, but full-time editors invested for the long haul are the core of the project.

Conclusion

The instincts at PAL and ALPLM are good. The staff vision statements—particularly those

from Springfield—laid out some very workable and practical solutions. Trust that and treasure

that. New leadership doesn’t need to deviate from that core sense of what’s right, it just needs to

communicate that renaissance to the world outside the PAL office suite.

As I have said before, PAL was the most influential advisor that CWGK had when our

project was getting off the ground. That vision—however imperfectly manifested—has become a

13

core vision for us as well. It has been a distinct honor to have the opportunity to pay back some of

that investment in us. You always have friends in Frankfort.

Patrick A. Lewis,

Civil War Governors of Kentucky

1

The Papers of Abraham Lincoln

Review and Planning Team

Site Report

Susan H. Perdue

15 March 2017

2

Section I: Current State Mission The Papers of Abraham Lincoln (hereafter PAL) was formed in 2000 with the following mission statement that can be found on its website: “The Papers of Abraham Lincoln is a long-term project dedicated to identifying, imaging, transcribing, annotating, and publishing all documents written by or to Abraham Lincoln during his entire lifetime (1809-1865).” The project’s mission statement puts it squarely in line with the prevailing model of modern documentary editing. That model was established by the papers projects of the Founding Fathers which began in the mid-twentieth century (e.g. Thomas Jefferson, John Adams, and Benjamin Franklin. George Washington started in the 1970s). The distinguishing feature of the modern documentary edition is its claim to be authoritative and comprehensive. This rests on the premise that the edition publish all of the known incoming and outgoing correspondence from their primary subject. This was a marked departure from the nineteenth century documentary edition which included only outgoing correspondence from the main subject. As a result these early editions represented half of a historical conversation. Perhaps more egregious was the tendency of nineteenth century editors to abbreviate published texts, to redact sensitive material, and to eschew the use of back-of-the book indexes that would assist users of the edition. It is important to acknowledge that at this same time—in 1953 to be precise—an effort was begun to publish the “collected works” of Abraham Lincoln. Begun by Roy P. Basler and sponsored by the Abraham Lincoln Association, this multi-volume set of correspondence, speeches, and other writings was completed in five years-time. Basler was not an authoritative edition as defined above. At eight volumes it could not have represented the entirety of Lincoln’s documentary legacy and it likely did not represent a comprehensive search for all of those documents. Although it was a significant scholarly effort it was, and is seen, as needing to be replaced by a modern documentary edition. The Basler edition was, however, a success in its adherence to the original mission statement and timely completion. It is still widely used today in the absence of an available modern scholarly documentary edition. Each of the documentary editing projects mentioned here began with a mission to replace an earlier version of itself and to become the final authoritative version. The Jefferson Papers at Princeton, for example, states that its volumes are “designed to supplant the four highly selective and unreliable compendiums of [Jefferson’s] writings published between 1829 and 1904, and to ensure that the task will not have to be redone.”1 This is an admirable mission and I understand the desire by editors to believe that their work is the final word, but I think that we run the risk of historical shortsightedness to believe that the work is ever complete. That is, documents will continue to surface in the marketplace and in the archives. Who’s to say that in 200 years-time another edition might not be called for? Understanding and even embracing this mindset will be useful for the Lincoln project going forward. The projects zeal to be seen as the ultimate and

1 https://jeffersonpapers.princeton.edu/about-us/project-history (accessed 2017-03-10)

3

final Lincoln edition explains a project gestalt that sees the document search as never ending and the editorial process as complete only when every known stone is turned, not once but multiple times. The project has lost sight of the original mission. Scope With the advent of modern documentary editing projects, large scale searching for documents was begun in repositories throughout the world. For example, the Papers of Thomas Jefferson has collected documents from approximately 750 repositories worldwide. This and other statesmen projects have assembled document archives that are similar in number to that of PAL (ca. 100K documents). PAL is significantly different from these projects for several reasons. First, PAL was conceived of as a born digital project and therefore it does not collect paper copies of documents as the other statesmen projects do. PAL scans all of its documents in their entirety at a uniformly high rate (600 DPI) and quality. Second, PAL promises to publish transcriptions and manuscript images. This approach has become more common in the last 5-10 years but it has not been the norm, primarily because projects did not obtain permission to publish manuscript images, only transcriptions. That was the only sensible approach in print publication because of space limitations. Digital publication has dramatically changed things in this regard. Finally, most projects adhere to some form of selection while being described as comprehensive. That is, routine documents might be calendared, abstracted, or described in annotation. This is especially true of enclosures. PAL has adopted a policy that can only be described as unconditionally inclusive which is pushing the boundaries of reasonable expectations. PAL’s approach to fulfill its mission of digital publication in terms of these three objectives should be revisited. Some suggestions will follow in the Future State Section of this report. Over the last decade the project has expended approximately $3.5 million on document searching. During that time slightly more than 68,000 documents have been found. That averages out to about $52 per document. That dollar amount does not seem terribly high yet I must question the advisability of the overall expense given the fact that each of the 68,000 documents could not be assigned equal weight in terms of its historical importance in the overall edition. And numbers of documents do not tell the whole truth. It is more accurate to talk about pages when trying to get a handle on the volume of material in play here. Some documents are one page long but many more number higher than that. (In our Skype conversation with David Gerleman he noted that one court martial case ran 2,500 pages—all of which would be considered in scope and thus receive the full treatment). This example and thousands more, represent documents that are neither to nor from AL. He might have endorsed a wrapper or a clerk might have referred said document to him. The fact that all of them receive the full treatment: high quality scanning, accessioning, multi-level encoding, transcription, etc. is simply untenable. Clearly, the document searching process is out of control. There is no rule of reasonableness and no serious review of the return on investment. Counter intuitively, the staff who are conducting the search are given numerical goals each year rather than numbers of document collections to search. Their intense need to find a certain number of documents in that time frame leads them to expand the scope in order to be more inclusive with the document search. With another decade of searching being suggested, the project will only exacerbate the glut of documents which will likely be increasingly marginal in scope. No temporal or content limits have ever been applied

4

and in fact, the scope has only been expanded over the years leading staff to feel unsure of what they might have missed. At one point some of the staff expressed their concern about the expanding scope of the search leading to a vote on whether to change the policy. Without a clear majority in the vote and no consensus there was no adjustment in the scope of the document search. Because staff do not know how many pages each document represents they are unable to answer questions about the length of time it takes to process their finds. Past promises of unlimited funding and time made to the same staff have fostered a strange sense of disconnect in working to meet targets that will never be enforced and that are not tied to a larger strategic plan.2 The volume of material being brought in from the National Archives searching is significant. The notion that the project must include materials that are readily available online in digital format, such as the Congressional Record, is outlandish. Enclosures such as books, reports, and messages are typically available in full on the web. (Staff resisted a push to include entire books received by AL at one point, an initiative which was “voted down”). Documentary editing projects do provide full bibliographical information on such enclosures but they do not usually print them in full. The decision to include something in its entirety is made based on its uniqueness or rarity (e.g. handwritten or unpublished items), or some direct intervention by the main historical actor such as marginalia or commentary, Steady State It is useful to briefly describe how a documentary editing project best functions. This is based on my experience at several such projects including the Papers of James Madison, John Marshall, and Thomas Jefferson Retirement Series. These projects have a reputation for steady and on time publication. Every project has some personnel issues and dysfunction, much like any family, but by and large, they find ways to move the publication process forward each year. Documentary editing projects are engaged in a publication process that never stops. That is, they follow a cycle of publication from start to finish that has multiple deadlines which require absolute team work. In order for the work of a documentary editing project to achieve a steady state and avoid the "crunch times" and stresses of these deadlines, projects must be managed to achieve a steady state. Each person on the team has a role to play in this process. Although the editor's role differs from the fact checker's, each person must perform what is expected of him/her and then some. All staff members should be clear about what he/she is assigned but also be aware of the tasks performed by everyone else. It is clear that there is no sense of teamwork or shared purpose at PAL, other than a sort of opportunistic collaboration that has evolved in recent months in a management vacuum. At this juncture it is important to note the role of the institutional host in the life of a documentary editing project. Most projects are affiliated with an academic institution through which grants are submitted and managed, benefits and office space provided, and IT support delivered. In turn, the project editors might teach courses, hire graduate students and interns, or contribute to archives and special collections. Fewer projects are housed at museums and

2 Surprisingly, the staff has not done a comprehensive search for AL materials in newspapers. That would seem to be a fruitful source especially for letters that they might not have.

5

historical societies. Some noteworthy projects in this latter category are the Papers of Thomas Jefferson: Retirement Series at Monticello, and the Adams Family Papers at the Massachusetts Historical Society. These projects share close relationships with their host, contributing to the scholarly and interpretive mission of the organization and working collaboratively to raise long term funding. In fact, the projects and their strategic planning are folded into the larger organization’s strategic planning. Development staff members work with the project directors to build wasting endowments such that the projects are funded through to completion. It is a mutually beneficial relationship that works very well. Not surprisingly, universities operating on a much larger scale with hundreds of ongoing grant-funded projects tend to pay less attention to the documentary editions in their midst. It generally falls on the projects themselves to make the case that they are important to the academic mission of the university. Whatever the relationship may be, the project’s success relies on a well understood and symbiotic connection to the host institution. PAL’s relationship to the museum seems to be trending in this direction which is a very good thing. That will be absolutely essential for the project to obtain a steady state. Project Organization PAL’s lack of a clear and well understood strategic plan has hampered the project and it impacts every phase of the operation. Every aspect of the publication should be aligned to the plan: document searching, editorial methods, publicity, and fundraising must reflect and be responsive to the larger goal. There is no apparent short term or long range planning. There is no plan to integrate the legal papers, the Lincoln log, or the National Archives images with the overall PAL content.3 The project’s publication of a site featuring images from the National Archives and the Library of Congress (http://papersofabrahamlincoln.dataformat.com/) is symptomatic of the larger issues. The site is described as a “publication” that “provides early access to images and basic information for documents located and scanned” at the two repositories. It is a “preview” of the future publication” and “reflects the [project’s] effort to locate, scan, transcribe, annotate, and publish electronically all documents written by or to Abraham Lincoln during his lifetime.” Yet there are no transcriptions, no annotation, no advanced searching, and no subject headings. None of the editors’ labors are represented in this content (other than the basic information of title, date, and collection). It is not even a taste of what’s to come but a pallid version of their hoped-for digital edition. In this case, the funders pressured the project to put something online. This site was not in line with what the project wanted to do, but they did it to assuage their funder. Clearly the staff felt this “publication” was unsatisfactory, embarrassing even, something they described as a “vanilla site.” This is because the content does not showcase any of their intellectual efforts which they spent more than a decade on. Thus they feel little pride in this achievement. Perhaps that is why it is relegated to the sidelines on the website where no one will find it. Staff The project is divided into two separate staffs, the primary staff residing in Springfield and the secondary scanning staff in Washington, DC. This bifurcation of staff is reflected in the distinctly different jobs that the two teams are engaged in. The rift between staff members in the

3 Oddly, these three major resources are presented as “Related Links” from the homepage of the PAL website, rather than featured as AL content (http://www.papersofabrahamlincoln.org/ [Accessed 2017-03-11]).

6

two locations is to my mind the most challenging aspect to be addressed. There is a serious disconnect between the Washington arm that is collecting material and the Springfield arm that must process it. Although they are working toward the same goal (which I would argue they don't really share), they are at cross purposes. Indeed, it is also apparent in the “Staff Visions for the Future” that they wrote after our site visit. The comments about future state from the DC team were very narrow in scope, confined to the searching, whereas the Springfield staff members put together more detailed scenarios for a future state that reflect their greater integration with the editorial work of the project. The DC staff, who share the same professional credentials as their peers in Springfield, are assigned a tedious process of searching collections with ever expanding objectives balanced with a scanning process that is totally out of their hands. These factors have led the scanning staff to find new ways to engage their intellect and provide them a sense of active participation in the edition. This means they over reach in their inclusion of relevant documents and they start to edit and change documents in the CMS, in a manner that creates problems for the home office. Moreover, the DC staff as a result of their total lack of editing work are unable to find jobs with other documentary editing project. Arguably, they are not even qualified to work at their own project (PAL) when they transition off scanning to the home project, because they don't fully understand the editorial apparatus. Without question, they have a difficult time finding positions at other documentary editing projects if this is their only work experience. I know this from firsthand experience having interviewed previous PAL staff members who were searching for positions at documentary edition projects where I worked. These candidates were not competitive with other applicants from projects where they had even a minimal amount of experience on editorial production (i.e. research, fact checking, and annotation). Just as problematic, if not more so, is the overall demoralization of the staff. I mentioned dysfunction before in the ranks of documentary editing projects--and of course, most office environments have a level of this. In my thirty years of work, I have never witnessed a group of people so shell-shocked, so unsure of their own abilities to do the work, so unable to figure out or their priorities, and so uncertain of the future. This probably comes as no surprise to senior staff. At the risk of getting ahead of myself, I urge you all to focus on the staff that remain as your starting point, well before you hire a new director of the project. As noted above, in order for a project to meet its publication deadlines each year, the staff must work together as a team. There is a balance of tasks across the staff and everyone needs to feel like they are part of that team. PAL staff members are isolated from each other because of their geographical separation and a “need-to-know” mindset that has marginalized everyone and created siloes of expertise. This does not begin to address the degree to which the project is divorced from the parent institution, the Abraham Lincoln Presidential Library and Museum (ALPLM). Although they are called on to provide reference assistance to the library, there appears to be very little integration between PAL and its host institution. A failed attempt at establishing a social media presence by staff members is a good example. One staff member started a Facebook page and was chastised because it wasn't part of the museum's social media efforts.

7

The project’s affiliation with the University of Illinois—Springfield is very problematic. There seem to be few benefits at this point in this relationship. Those staff members who are employees of the university are resentful of their employer which does not appear to support their work and does not pay them an equitable salary. The benefit of this affiliation going forward is questionable. To my mind it does not seem to be necessary for the work of the project. Content PAL uses an XML content management called PubMan, a system that is also used by several large documentary editing projects. Those projects use PubMan for both print and digital publication because the XML gives them the flexibility for digital and print publication from a single source file. PubMan has some limitations in terms of accessing the content and it requires some practice in building complex search queries. Amazing to me is the degree to which the staff is not comfortable using the system after being in place for more than a decade. It would seem that expertise with the system and some of its internal workings was only shared with the staff on a need-to-know basis. (This is not the only area in which that is so--the question of document storage and institutional relationships related thereto seems a total mystery to the remaining staff). PubMan is an effective system and is successfully used by a number of projects as mentioned. The original decision to adopt such a system and employ XML was a good one and it puts the project in a better position in terms of next steps. Yet, there is an overall sense among the staff that they lack of control of their content. One of the biggest issues here is that all of the 100K plus documents are stored in one large batch (i.e. Title). There is no further breakdown of this content into logical groupings such as year, series, or document type. Any of these categories would provide a degree of intellectual control. Even as a starting point, putting all of the documents to and from Abraham Lincoln, would give the editors a sense of their key corpus of documents. Staff, through no fault of their own, were vague on the location of important documents scans and why they had ended up wherever they were. They also could not readily answer questions about the document searching at the National Archives. Was there an inventory of collections searched at the National Archives? It seemed that the project director had controlled that information. Lastly, the presence of 20-30 empty file cabinets in the office which were supposed to be filled with paper copies of digital documents was downright strange. Is there a need to replicate the archive with paper flow? Is there a method for capturing and storing related research materials? None of this had been worked out as the project has never moved into a fact-checking and review process. Workflow and Editorial Methods The RAPT team was provided with an eleven-page orientation document which appears to be the project’s workflow guidelines. The workflow document includes multiple references suggesting that there is inconsistent application of the guidelines, in part because the markup is excessive and it has been difficult to get staff buy-in on the entire apparatus. But the issues seem to run deeper than that. Comments in the document observe that staff were unaware of some policies. Conversely, staff reported that the project director was not involved with policy creation and himself seemed unaware of the project’s methodology. This runs counter to procedure in most documentary editing projects where the project editor sets the editorial policy and is the ultimate decision maker when there is disagreement among the staff. PAL’s project director seems to

8

have applied a loosely democratic process in order to arrive at consensus which when it did not happen, his response was to double down on the approach. From the outside looking in this appears to be a means of punishing the staff for expressing their views. In the same way the scope of document searching expanded over time, the editors expressed dismay over the fact that the editorial policy was constantly tinkered with and changed. Any such change creates a ripple effect across the project forcing editors to double back and redo things that were considered to be complete. Given the sheer number of documents and the extensive encoding required of them, this was very problematic. As one staff member put it, the work is “exhausting.” One is “always up against the numbers. Anything you do to the document is great, but the tension remains. There is no pacing. The pace is a grind. Your only metric for success is the numbers. You can't hit the number because it's untenable.” In an ideal world, the workflow and publication pace is steady year to year. Consensus is great but ultimately the project director needs to make decisions about editorial methods and annotation and explain the rationale behind it. If one looks at the project by the numbers one reaches some astounding metrics. In 2012, the project estimated for the purposes of a video presentation, that each document takes nine hours to prepare it for publication. Given the total number of documents, the fact that the project continues to find more, and the exhaustive level of encoding and annotation, the project will not be complete for 99 years. This is unsupportable. The editors do not work as teams and no clear distinction exists between staff member responsibilities. Volunteers are transcribing documents in XMetal and also adding encoding, which is arguably a form of annotation that they should not be doing (and which should be added at a later stage). Editors are transcribing, in some cases for years at a time without editing. Workflow is adjusted to accommodate new deliverables and deadlines. In October 2016 staff reports that the solution for getting a set of 2,500 documents ready was to remove the review and fact checking stages. The staff disagreed with this approach, but the fact that it was even suggested is shocking. This is indicative of the inordinate emphasis on encoding over establishing the authoritative text and factual accuracy. Not to mention the problem of a project jettisoning some important stages of the process in order to meet a deadline. The document lays out the criteria for document scope (what’s in play) as well as an extensive list of document types (while noting that there have been invalid document types added to the collection). Strangely there is no discussion of editorial method for the actual text itself. How does the project deal with superscripts? Special characters? Marginalia? Overall it seems like there is very little focus on establishing an authoritative text given that there is only a single visual comparison made by an editor. This is described as a “single proofing” but it is not truly a proofing but a verification. The project is missing a tandem proofreading (two people reading, one from the manuscript and one checking the transcription), a workflow stage that is critical for establishing reliable texts, in my opinion. There is too much weight focused on the markup, indexing, and annotation in the workflow in general.

Encoding and Annotation The overwhelming focus of the project’s workflow is on encoding in the realm of what is often called semantic tagging. That is, the project is using encoding and normalization of terms to

9

define people, places, and events. This is certainly a worthwhile endeavor and not one adopted by most documentary editing projects, at least among those that even perform some level of encoding within the text. But, it appears to me that this process has become the tail wagging the dog. That is, it has expanded beyond the project’s capacity to implement across all of the documents. And it has been done at the expense of providing authoritative text and even publishing said texts. It is applied without inquiring as to its usefulness for the end users. Finally, there is no acknowledgment at any point by the project that because they are publishing a digital edition, they can apply these processes iteratively. In the digital realm, content can be updated, corrected, and republished.

Here are a few examples of encoding that has gone awry:

Stage 2 markup: encode persons, places, and organizations every time they appear. Why is this necessary? It might make more sense to encode the start and end point which signals to the user that the subject or person is dealt with throughout the selection (e.g. the way an index works as a signal to users that they must read the entire selection), but encoding every instance is overkill. This is especially true when one considers that every one of those encoded references must be linked to a master record, multiplying the work exponentially. The question needs to be asked, what purpose does this serve?

Stage 2 markup: abbreviations are tagged every time they appear. Why? What purpose does this serve? It is not necessary for searching as search engines can be provided language to facilitate hits (Lieut. can be given an equivalent of Lieutenant). Abbreviations abound in historical documents. Certainly there are times when an editor needs to explain what an abbreviation might mean in annotation (and in the encoding here), but a rule of reasonableness needs to be applied.

Stage 3 markup: every signer of a petition is considered an author. The implications for this decision are immense. This means that every signer is tagged as an author and the handwriting tag is added. It also means that each signer is provided a biography. Cannot the assumption be that the signer is the author and that the editors only signal when that is not so? Encode the exceptions not the rule.

Subject indexing: I admire the project’s efforts to implement a standardized subject terminology such as FAST which I have used myself. The issue here is that the editors cannot visualize how they have implemented it across the content. As noted in the manual, it has been “implemented inconsistently” and is “nearly useless.” I think it is probably more useful than they might imagine but they cannot get a handle on how it has been used. This terminology needs to be applied in a more centralized and visible manner—just as the names and places are in PubMan. There are no clear policies in place about how it is to be applied except for some agreement that two levels of subject indexing are the standard. In this, as with so many other areas of the project, if the staff perceive a task as useless there can be no positive reward for accomplishing this task.

Annotation: Encoding is annotation. That is, the person doing the encoding must make informed historical decisions about who the person is or what the event being described is. The project’s separation of annotation from the earlier stages of encoding is a false one. The editors should be making decisions about the level of semantic tagging alongside the annotation. They should work together to establish the historical context of the document and how it will be retrieved by users. The manual describes conflict and inconsistent application regarding the level of annotation. Staff explained that originally the director said there would be no annotation, yet the

10

policy has ballooned over time. This is not surprising. Below are a few of the issues with the annotation policy.

Legislative history from “inception to end”: This is excessive annotation. Staff observed that explanatory notes has “gotten out of hand” and we were provided examples of legislative histories that no user would probably ever read.

Provide enough context for the reader to have a basic understanding of the document’s place in Lincoln’s life. Yes, this makes sense for the Papers of Abraham Lincoln. “And in American history” No, this does not make sense with the exception of a few seminal documents such as the Gettysburg Address.

Guidelines about writing biographies of people and events are also in play—that is they have expanded over time and are the source of internal conflict.

Tagging within annotation: what is the use case for this? This content can be retrieved in searching because the annotation is in the editorial voice. There is likely to be some overlap in the annotation with the document itself. The process of encoding annotation is again, really time consuming and in my mind has a limited return on investment.

Workflow stages: review and fact-checking. There is almost nothing said about these critically important stages which is eye-opening.

Section II: Future State The project is at a critical juncture. I am impressed with the management’s current approach to take some time for reassessment and review. Along with the process that the RAPT team is engaged in, I recommend a process of strategic planning that includes management and staff. I also think that the new director might want to be involved in this process but whether that’s possible or not, is unclear. It was a very good idea to request the staff to put their thoughts on paper regarding the future of the project. Strategic Plan The PAL mission statement still holds true and is worth revisiting by everyone on the project. Are there any founding documents that could be reviewed by staff to refresh the sense of purpose and foster a sense of collective mission? It might be useful for the staff and management to discuss the impact that the Legal Papers project had on the origins of PAL and the zeitgeist that was inherited from that project. There is an obsessive fixation on the task of finding documents, which I think is expressed in the assignment of “The Lincoln Corp of Discovery,” an “elite group of project staff members” who have found Lincoln documents. Everyone on the project must feel like they are on the same mission of discovery and are equally rewarded—not just for finding them but for publishing them. The project mindset needs to shift toward publication. What does that mean? Has the staff ever engaged in a conversation about their ideal edition?4 The project should begin with a discussion that includes all PAL staff and management about

4 We were provided with statements, or “Staff Visions for the Future” from four of the PAL staff on 3-13-17 with their thoughts on the project moving forward and there was some discussion of editorial specifics, but very few specifics on their hopes for the digital edition in those documents.

11

what this would look like. Engage in some blue sky thinking. Create some mock-ups of the site and how documents are presented with their images. How is encoding presented? How do users search the content? What values are filtered? What does the digital edition do that a print edition cannot do? This costs you nothing to do but time and is a necessary first stage for any digital edition. It will help foster staff comradery if nothing else. All of these decisions will help inform and guide the issues below. Include in your discussion the other content you have. How might the Law Practice content be integrated? What about publishing the document inventory online as a finding aid for users? Why the strict adherence to phases of AL’s career for publishing? What about breaking documents out of series designations? Allow the staff to take this discussion in tangents as needed. Certainly they have earned it, but try to steer it back to basic principles of the edition. At this point, it is critical that the project take the conclusions from the strategic planning process and mold that into a publication plan. The project must decide on an outline of the digital edition and how and if, it will build from the current project content management system. Or will the latter be reconfigured somehow? What would that imply and what costs will be associated with that plan? The project has built a solid infrastructure based in an XML environment which is a very good starting point from which to work. The necessary next step is determining how the content will be presented online which will determine every other step that follows. Certainly a digital edition is the expectation of the PAL project, but it is also worth discussing whether some form of select print publication could be a by-product. The overarching goal is to look at the full trajectory of the project from this point forward. When will the project be completed? No project can be totally open ended. A publication plan must be prepared that fleshes out the details of how many documents published and when, and the only what that can be done, is by estimating an end date for the project. What will finishing the project mean for the staff and for the host institution? Is there another project that the editors could move into that might support the mission of the Library and Museum? Importantly, how does the document searching continue going forward? If it is scaled back or modified, how long will it take and how does that align with the publication plan? Scope The project needs to realign itself and reassign priorities to all stages of production beginning with document searching. If resources are finite, which they always are, some decisions have to be made regarding the assumption that all documents receive the same level of treatment. The project has to work through a triage process to assign some priorities for the content that can be applied consistently by staff and are well understood by staff. Document search: this has expanded beyond what’s necessary. Level of relevance should be applied to the document search. Referred documents that were sent to AL as routine or presumed to have been sent to him could be summarized with metadata only and/or digitized at a lesser DPI. An all or nothing approach to obtaining digital images does not make sense. Reasonably good digital images for documents, especially those not to or from AL could be obtained without the quality now demanded. Images of lower quality could be acceptable.

12

Editorial methods and workflow have also expanded beyond the project’s capacity to keep up with the content. The encoding scheme should be scaled back based on the outcome of the strategic plan and what is feasible within a structured publication plan. Project organization Staff The staffing model needs to be revisited. As noted above the separation of the staff into a searching team (DC) and an editing team (Springfield) has set up a tense dynamic. Everyone on the staff hired as an editor should have capacity in his/her job to perform all of the editorial tasks. PAL needs to find ways to bring the DC team into the fold, starting with their inclusion in the strategic planning described above. Regular meetings with the Springfield staff would help here. More transparency between the two teams would help. Revamping the document search will have an impact on the DC staff and their long term trajectory. How do they see themselves in five years? Will they move to Springfield to join their fellow editors? Staff morale needs fixing. As a starting point, more focus should be paid to professional development. All editors should be enabled to perform all stages of the work, especially annotation. They should be encouraged to participate in the annual meeting of the Association for Documentary Editing and the Editing Institute. They should be encouraged to write articles for scholarly journals on their finds and their work process. They should consider talking about the project at digital humanities and historical conferences. I suspect very few documentary editors, for example, know about the complex encoding that is being done in the PAL documents. Their colleagues would be interested in seeing what is happening at the project. With a realignment of goals and the possibility of dividing content in smaller batches, perhaps editors could be assigned documents for which they are solely responsible for both encoding and annotation. Editors might also be assigned batches of documents based on a correspondent or a theme running through a set of documents. These are all ways to enable an intellectual connection with the material and elicit the best research and insights from the editors. They are not automatons but scholars. The notion that an editor is resigned to transcribing documents for five years while waiting for something better to come along is deplorable. Yes, everyone should have experience with all of the processes from the mundane to the fun stuff, however, there needs to be some recognition of their scholarly credentials in the work. The configuration of future staff depends, in part, on the publication schedule. Without question the project requires additional support staff to perform fact checking, review encoding accuracy, and other pre-publication steps. These workflow stages appear not to have been executed to date so there is some flexibility here to shape the positions and maximize the efficiency for the benefit of the project. If the editors in DC are allowed to contribute to the encoding and annotation, it seems to me that the project could launch a publication schedule without a significant ramp up of the number of editors. This means two additional FTEs would be required: the project director, and a research assistant. Given that the current set of editors are all assistant editors, promotions to associate editor should be considered for those who have worked at the project for more than five years. I don’t know if previous editors were designated as associate editors, but at this point it would be a huge boost to

13

current staff to recognize their diligence and dedication to the project by creating another tier for the editorial staff. (John Lupton rose to the level of Associate Director/Associate Editor in 2006 after working at the project since 1999). Advisory Boards: during the site visit we discussed the advisability of an advisory board for PAL. My impression is that the current board has divided loyalties to the project director which is problematic. I think a board comprised of documentary editors and historians is a good idea. Whether such a board would have forestalled the current situation is hard to tell, but fellow editors with experience on their own projects would be invaluable to the project director. They might only be consulted on an annual or bi-annual basis and there might only be a single site visit to kick things off. Content The guidelines determining scope for PAL documents should be revised to reflect their relevance. Everything to and from AL is top priority and should be treated as level one documents, if you will. Enclosures and referred documents (for AL’s eyes) might be treated as level two. Third-party documents, documents in which AL is quoted or described might be treated as level three documents. Third-party documents are often included in documentary editing as part of annotation for they provide useful context. Sometimes they are even included in full in an edition if they provide insights to character or behavior of the project’s main subject. However the relevance is assigned, it can be used to guide staff in the varying levels of treatment—from comprehensive in the case of AL documents—to less so in the case of third-party documents. Such an approach will save time and resources and enable the staff to keep its focus on the main attraction. Editorial projects rely on rules to guide staff in day-to-day decision making. That said, it does not make sense for projects to follow rules without allowing exceptions. Some third-party documents will rise to level one importance in which case they would receive the full treatment. This is a decision that can be introduced by the editors themselves which gives them a sense of agency, but the decision will ultimately be the director’s. Whatever the case, the policies must be flexible as must the people enforcing them. I urge the project to give some reconsideration on the organization and presentation of content. To start with, the documents in the PubMan system should be broken down into smaller titles if nothing else to make them easier to manage and deal with. They might be broken into decades or even single years. Likewise, the adherence to series designations seems to be hampering rather than moving the process forward. The idea that an entire phase of AL’s life must be released at once (e.g. ALL of the legislative documents) rather than subsets of the group prolongs delivery of material. Remember that documentary editing projects that publish print volumes only publish 500-700 per volume with one volume, on average, coming out each year. That is steady state for a documentary editing project. Here PAL can follow suit. Why not a document roll-out on a bi-annual basis of 500 documents? Those documents could be drawn from multiple series---legislative and presidential. Users will benefit from a wider chronological set. Users do not care that you have series designations or that your groupings of documents are bound by your dating. They care about getting the material. The longer the project delays in its efforts to assemble large well defined batches, the longer the public must wait.

14

Workflow and Editorial Methods The project’s workflow should also be revised according to the scaled-back objectives on encoding and annotation that would be imposed as a result of the strategic planning process. The workflow must respond to the renewed emphasis on publication which means much more time devoted to confirming the text through oral tandem proofreading, sense reading, and fact-checking. Editing is never all or nothing. The editors need to be empowered to make well informed decisions on their own regarding the level of encoding and annotation, based on their understanding of the wider corpus and the historical context. The whole point of their job as editors is the weighing of the need to explain something to the end user against the need to get the material published. Encoding and Annotation Encoding stages 2 and 3 markup should be considered in tandem with annotation. They are a form of annotation. As the editors assess each document for the level of annotation required, he/she should also be looking at the degree of encoding required. This includes all of the semantic markup: names, places, organizations, subject headings, abbreviations, special characters, and special formatting issues. Also at this stage the editor should be doing a second verification of the text just as a matter of course while applying the markup. Technical Considerations My summary here has provided ample evidence of numerous questionable decisions by the project director, but I must give him credit where credit is due. His decision to adopt a robust XML content management system and TEI-based encoding scheme was a good one. It was made for the right reasons: XML is the best way to approach a digital edition of this scale because it is an agnostic language that can be repurposed and reconfigured. TEI is the de-facto encoding protocol for historical and literary editors and it has great currency and uptake in the profession. The project is in very good stead in this regard. And the fact is that past over zealousness across the board is less a problem for you now than not enough. Things do not need to be redone but rather scaled back. This is entirely possible because the project is using XML. What’s missing is the path to publication. But once that is figured out, some of the excess found in the XML encoding can either be ignored or employed over time if so desired. The fact that the publication path has not yet been determined is one of the major missed opportunities of the former director. It could never have been the plan to present the documents to the public using PubMan, a system that is not at all designed for a front facing site but rather a work horse system that does the heavy lifting of document management, versioning, and storage for the project. There may be a perception at the project that the system is broken because you cannot do anything with the documents. The problem is that you only solved half of your publication challenge. What the project requires is a publishing process that will move XML documents out of the CMS and onto a website. What this system is will largely be determined by the requirements fleshed out in the strategic planning stage described at the beginning of the Future State section. Therefore my intention here is not to give recommendations but lay out some considerations and some possibilities based on my experience.

15

There is nothing wrong with the current content management system in use at the moment. PubMan is operational and supported by the vendor. Other documentary editing projects continue to rely on it to publish both digital and print editions as we speak.5 Note that the vendor responsible for PubMan (IDM) is actively working to build the next generation system designed to replace PubMan. I have some knowledge about this new system and have been keeping up with it, as well as advising the folks at IDM as they put this system together. It has been useful to them to have ongoing input from the documentary editing community. Their plan is to roll out this replacement early this summer. So more to come. The replacement for PubMan will not provide the web-based publishing side, however. That still needs to be decided upon. I just want to be clear here that the project should not dismantle the PubMan system out of a sense of panic or impression that other systems will do what is needed and more. There is no system “out of the box” that will do everything the project needs at this moment in time. And the cost to dismantle the system and move it into something else will be costly and time consuming. This is not a decision to take lightly and frankly one that makes no immediate sense. I have had extensive experience using PubMan over the last seventeen years, both at the Jefferson Papers: Retirement Series from 2000-2008, and more recently I used it for a project I directed called Early Access from 2009-2015 (see https://founders.archives.gov/about/EarlyAccess). The only way to manage a project of this scale: 55,000 documents produced for digital publication in 3 years-time with 40 staff members, was a web-based system like PubMan. That said, it has always been a frustration to me that one could do very little with the XML itself without experience using XSLT (Extensible Stylesheet Language Transformations). It has been a goal of mine for the last several years to find a web-based publishing platform that would enable people without programming skills to build a quick digital edition. In the summer of 2014 I assembled a group of folks in Charlottesville (editors, publishers, funders, programmers), to discuss just this problem: the production chain from content creation to publication on line. You can see more on the desiderata that came out of the meeting in the Production chain inventory document at the end of this report. It might be of interest to you. The technical solution that comes closest to providing the full production chain is Islandora, an open source technology stack that provides a full suite of tools to do most of what is required for a digital edition. Essentially, Islandora is built on the repository store Fedora, with a Solr search engine, and a Drupal CMS. I have now had experience building several projects using Islandora. I use the term “building” loosely as I did not build them at all but rather am using hosted solutions for all three sites. The vendor is DiscoveryGarden. They provide hosted solutions that have a monthly cost and they can also provide help in designing the website and moving content into the site. You can also see the site I am currently working on which is an institutional repository at the Virginia Foundation for the Humanities called Discovery Virginia (http://dv.saas.dgicloud.com/). This is built in Islandora.

5 The Jefferson Papers at Monticello uses their PubMan system to publish family letters to the Monticello website. The editors hired a programmer/firm to build the transformation of their documents to move to the Drupal CMS in use by the Monticello web master. See http://tjrs.monticello.org/

16

The project that is most like PAL is the Patrick Henry Digital Edition (http://www.patrickhenrylibrary.org/)

By that I mean it is a documentary edition in which document transcripts (TEI-based) are featured alongside the manuscript image. Transcriptions are created in the XML editing software program Oxygen and uploaded to the site. Islandora has many great features especially its reliance on standardized metadata schemas such as Dublin Core and MODS (metadata object description schema). There is more that can be said on this platform and its possible application for PAL but I am not certain that it is the direction PAL should go right now. As I mentioned above, to move the PubMan content into Islandora will be a massive process with a big price tag. Your TEI structured documents will all have to be mapped into a Dublin Core and MODS environment requiring extensive conversion, not to mention the related entity files for people and places, to name a few of the challenges.

My colleagues on the RAPT team will no doubt provide very good examples of how other platforms such as Omeka and Drupal can be used very successfully to present content online. In the case of Drupal many programmers and web developers work with that CMS these days and the PAL project could absolutely go in that direction. That is, once the specifications are worked out and the XML is provided to a programmer, a web interface can be built to crosswalk documents to your site. Because you have your content in XML you are poised to make this happen. Each approach presents different issues and different costs. During a strategic planning process, PAL must make some determinations on the minimal level of publication that they want (what are the must-haves) as well as the features that are great (what are the could-bes), in order to guide the process. This will help the project formulate a detailed request for proposal that will guide the selection of your technical solution. This document is not that—that is I do not think that I or my RAPT colleagues is in a position to recommend your technical solution.

Final Thoughts The PAL staff has endured significant stress over the last year or so. Despite the challenges they have faced they persist in their commitment to the original mission of the project. I am confident that with some time and serious planning the project will be rejuvenated and find its footing again. I am sure that my RAPT colleagues will also provide you with good advice and insights that I have not thought of, and from all of these sources you can come up with a holistic plan. I am encouraged by the actions taken by the staff and management to make this happen. In an effort to end on a more positive note than perhaps characterizes this report, I want to commend the extraordinary amount of good work the current and past staffs have accomplished. They have traveled thousands of miles in search of AL documents. They have assembled a major collection of documents numbering more than 100,000, people records numbering more than 42,000, organizations numbering more than 2,000, and place names numbering more than 2,500. There is so much good content here that needs to get out to the wider world. Much remains to be done to put this altogether in a sensible, efficient, and meaningful way with a clear sightline. I am ready to offer my services in the days and weeks to come as you put your plan together and I look forward to seeing the Papers of Abraham Lincoln come to fruition.

17

Document Overview and Recommendations Series 2A: Roll outs could start with this group of documents which has the most work complete. The roll outs could be as many as 1,000 docs every six months, or an initial release of 500. Two new workflow stages (in green) are recommended to be added.

Time period

Total documents

Transcribed Single Proof

Stage2 Stage3 Subject Indexing

Annotation Oral Proof

Fact Checking

Review and sense read

1824 to 1837-04-15

2152 2152 2147 2148 2148 2149 1917 0 0 121

1837-04-16 to 1841-03-03

5192 5192 5164 5182 5182 5175 3154 0 0 173

Total 7,344 7,344 7,311 7,330 7,330 7.324 5,071 0 0 294

18

Production chain inventory

Requirements for a Documentary Edition Production Tool(set)  

This document describes core requirements for a tool (or a set of tools) that a documentary edition would use at each stage of production publish digital (and/or print) content. Stages of production:  

Control File → Production platform (CMS, XML) → Editing software → Conversion process → Publication 

 Control File (Also known as an “Editorial Database.” Defined as a “system of physical and intellectual control over the collected materials that ensures that none of the editor’s work will be wasted or unnecessarily duplicated.” See Guide to Documentary Editing http://gde.upress.virginia.edu/01B­gde.html#h2.4 and http://gde.upress.virginia.edu/02­gde.html#h2.2.) 

1. Must be web­based, to facilitate access 2. Should incorporate controlled vocabulary functions, to facilitate taxonomy­first 

editing 3. Should be easy to install, configure, and deploy, to reduce technological barriers 4. Must be able to import and export data, to prevent lock­in and to allow chaining 

together with other tools for production & publishing 5. May be limited to describing and organizing document metadata, but need not be 

limited from extending into other aspects of production (as described below)  Content Production Tool (If the Control File is a catalog of the documents, this tool facilitates transcription of the documents and all steps leading to publication) 

1. Should be web­based—again, to facilitate access 2. Should have content management features, with the notion of users and roles, to 

facilitate editorial workflows (i.e., taking content from draft stage through editorial review and preparation for publication)  

3. Should allow batch operations on groups of documents (e.g., ?), to prevent RSI 4. May be limited to editorial workflow management, but need not be limited from 

extending into other aspects of transcribing the documents and encoding them for publication (as described below) 

 Editing Software and Transcription tools (Not necessarily separate from the Content Production Tool above, but transcription and editing are important enough tasks to deserve their own list of requirements) 

1. Should offer WYSIWYG­style editing, to lower the barrier for contributors who lack XML editing abilities 

2. Should offer a two­pane view: one pane for the scanned image, and one pane for editing the transcription  

3. Should provide a path to soliciting crowdsourced contributions 4. Must support version control 5. Must support TEI XML 6. Must support project­specific encoding guidelines as defined by TEI ODD 7. Must support validation of TEI against ODD­generated schema files  8. Should offer easy, ideally contextual, access to project encoding guidelines, with 

widgets for applying links among documents and identifying other controlled entities 9. Must be able to pipe content (at various levels of a publication’s structural hierarchy) 

into transformation tools  Transformation Process (If the content is encoded using TEI XML, tools are needed to convert the TEI to the media­specific formats in which they will be consumed: HTML, Ebook, print­ and/or web­optimized PDF. Conversion to these formats also facilitates proofreading.)   

1. Must provide easy to apply XSLT and/or XQuery transformations 2. Must provide sane defaults for most projects, with ability to customize 

 Publishing (Transformation may result in a folder of files that are sent to a web server or a publisher. In which case the project is “published.” But publishing might be a continuous part of the editorial workflow, in which case the editing and publishing tools should be well integrated or compatible.) 

1. Should offer a web­based view of the publication 2. Should support basic full text search 3. Should offer easy upload of new or revised content to server 4. Should allow access to various media­specific formats of a publication (HTML, 

ebook, PDF), as well as the raw TEI. 

Pinsker

-1-

PapersofAbrahamLincolnProjectConsultantReview

PreparedbyMatthewPinsker

March2017

ExecutiveSummaryThePapersofAbrahamLincoln(PAL)isalong-termprojectofimmensesignificancetoAmericanpublichistory,onparwiththeFoundingFathersPapersandotherflagshipdocumentaryeffortsthataredesignedtosharevitalhistoricalrecordsfromournation’spastwiththehigheststandardsofeditingandthewidestpossiblepublicaccess.Initscurrentstate,however,PALwillstruggletomeetitsownnobleambitions.Acombinationoffactorshasconspiredtofrustratetheproject,althoughitisworthnotingthatmuchimportantworkhasbeendoneandvaluableworkcontinues.Yetnobodycoulddisputethattheprojecthasreachedavulnerablecrossroadsandthatitdesperatelyneedshelp.Onthesurface,thisreviewisbasedonasingle,intensivesitevisittoSpringfieldinMarch2017andafewadditionalweeksofcarefulreflection,butitreallycomesoutofdecadesofscholarlyinteractionwithPALinallitsiterationsovertheyears–fromtheearliestLincolnLegalPapersworktothecurrent“dualstaff”operation.AsdirectoroftheHouseDividedProjectatDickinsonCollegeandoccasionalconsultanttoothermajordigitalhistoryendeavors,Ialsobringtothisanalysisalongstandingandwhole-heartedcommitmenttotheideathatdigitalpublicationcanhavearevolutionaryimpact,especiallywhenitiswell-conceivedandmaintained.Withthosefactorsinmind,myreviewoffersthefollowingkeyrecommendations:

1. PALshouldexperimentwithanonlineandprintpublicationschedulethattargetsspecificaudiencesanddoesmorethanjustfollowastandardchronologicalseries

2. PALshouldmodifyitsexpansivescopeofresearchandcomplexeditorialprocedurestoreflectmorerealisticassessmentsabouttheneedsofitsprincipalaudiences

3. PALshouldtakeadvantageofstate-of-the-artdigitaltoolstofullyintegrateearlierand

relatedprojects,liketheLincolnLegalPapersorTheLincolnLog

4. PALshouldprobablykeepitsdigitalplatform,butworkhardtoadaptitforpublicuse

5. PALshouldstreamlineitsstaffingoperations,butexpanditsexternalnetworksofadvisorsandpotentialcontentpartners

Pinsker

-2-

Recommendation#1:PublicationScheduleThatTargetsSpecificAudiencesInourdiscussionsduringthesitevisit,itbecameclearthatanyacceleratedpublicationscheduleenvisionedbythestaffgenerallyrevolvedarounddifferentwaysofproducingachronologicalseries.PALhasidentifiedabout12,500documentsfromtheperiodleadinguptoLincoln’s1860electionaspresident,withnearlyallofthemsuccessfullytranscribedandsingle-proofed.Two-thirdsofthedocumentshavebeenmarkedupinXMLatwhatthestaffdescribesasStage2or3levels,abouthalfhavealsobeensubject-indexed,andjustoverone-fourthhaveannotations.Finerperioddistinctionsalsorevealthatnearlyallofthedocumentsthrough1841(theendofLincoln’sstatelegislativecareer)havebeentranscribed,indexedandmarkedupwithjustoverhalfofthoseearlycareerdocumentsfullyannotated.Hardlyanyofthedocumentsfromanyperiod,however,havebeenfullyreviewedandthereisreasontobeconcernedthattheindexingandannotationshavenotbeenentirelyconsistent.ThatiswhyIamalittleskepticalaboutembracingastraightforwardchronologicalapproachtoacceleratedpublication.ItmightwellworkfortheperioduptoNovember1860,butseemsalmostcertaintostallafterwards,oratleastafterMarch1861(Lincoln’sinauguration).ThecriticalperiodbetweenLincoln’selectionandinauguration(secessionwinter1860-61)hasalmosttwo-thirdsasmanydocumentsashisentirecareerprevioustothatpoint,andyetwhilethese8,000+documentshavebeenlargelytranscribed,therehasbeenalmostnoprogressyetatindexingthem,markingthemuporannotatingthem.Thesituationlooksevenmoredaunting,ofcourse,onceyoustarttocastthenetoverthefourwartimepresidentialyears.Sofar,that1861-65periodhasyieldedover84,000documentswithjustunder30,000transcriptionsandavastterrainofNationalArchivesrecordgroupsstilltobesearchedunderthecurrentexpansivescopeofresearch.Inotherwords,toenvisionanacceleratedchronologicalpublicationseriesthatmaintainsuniformstandardsandbuildsfromtheIllinoisyearsthroughtheWashingtonyearsisalmosttosuspenddisbelief.Itjustcan’thappen,atleastnotbyreasonablehumaneffort.So,mysuggestionistoconsiderexperimentingwithanacceleratedonlinepublicationschedulethattakesadvantageoftheflexibilityofthewebinordertotargetandengagespecificaudiencespriortoany“final”release,whetherprintordigital.Mynewconceptfora“series”publicationisathree-foldapproachthatwouldlaunchsimultaneouslywithinaboutoneyear:

• TeachingSeries–onethatinvitesselectedguesteditorstohelpcuratecollectionsofteachabledocumentsonnotablethemesorepisodesfromLincoln’scareer

• WikiSeries–onethatinvitesvolunteergenealogistsandlocalhistorianstohelpannotatesetsofdocumentswithacomplexarrayofindividualorplacenamesin“wiki”orcrowd-sourcingfashion

• TestSeries–onethatinvitesresearcherstoengageinerrorreportinganduserfeedbackfromacrosstheentirebodyofdocumentsthathaveatleastbeeninitiallytranscribed,indexedandmarkedup

Pinsker

-3-

TheteachingseriescanbeenvisionedasawaytopromotePALinAmerica’sclassrooms,usingtheexpertiseofscholarlyadvisorsorhistorian/consultantstoworkinconjunctionwithprojectstafftocurateoccasionalreleasesofsmallgroupingsofkeydocumentsaroundpopularteachingconcepts,likeemancipationornineteenth-centuryfamilylife.WehavemodeledsomethinglikethisideaattheHouseDividedProjectwithourLincoln’sWritings:TheMulti-MediaEditionwebsite(http://housedivided.dickinson.edu/sites/lincoln),whichorganizes150ofLincoln’s“mostteachable”documents(includingseveralnewlydiscoveredonestakenfromPAL)intoaWordpresssitedesignedtoengageK-12andundergraduateteachers&studentsintheprocessofCommonCore-alignedclosereadings.Thesiteencouragestheproductionofclosereadingvideosandoffersahostofsupportingmulti-mediatoolsforresearchingeach

document.AnotherhelpfulmodelcomesfromtheDigitalPublicLibraryofAmericaanditsuseofPrimarySourceSets(https://dp.la/primary-source-sets).APAL-organizedteachingseriescoulduseavarietyofadvisors(Lincolnscholars,pedagogyexperts,digitalhistorians,etc.)tohelpdemonstratethevalueoftheprojectinfiniteyetconcretewayslongbeforeanyfinalpublication

wouldbeready.These“guesteditors”wouldhelpPALstaffselectagroupingofdocumentsandthenassistinreviewingthemtoensurethatallwereproperlytranscribed,indexedandannotated.Theycoulddothebulkofthisworkremotely,butalsocouldbeinvitedon-locationinSpringfieldforbrief,collaborativeworkingperiods.Ultimately,theywouldbetaskedwithprovidingaconcisewrittenintroductionandperhapsalsosomevarioustypesofsupportingmulti-media(videos,podcastinterviews,etc.)thatwouldhelpframethecuratedcollectionasavaluableteachingtoolfordifferenteducationallevels.Naturally,thescopeofsuchmini-collectionswouldbequitesmallrelativetotheoverallproject,butthequalityoftheworkwouldbehighanditspotentialimpactcouldbevast.ItmightalsoofferavaluableoutletforsomereallycreativeworkbyPALstaffandarewardingwayforthemtoconnecttheprojectwithoutsidescholars.Therewouldbesomecostsinvolved(honoraria,travel,etc.),butconsiderthe“teachingseries”asakindofnecessaryproof-of-conceptfortheultimateworthofthelong-termproject.ThewikiserieswouldaspiretoputtheenergiesofgenealogistsandlocalhistorianstoworkinwaysthatmightalsosupportandpromotePALbyprovidingthemwithapipelinefordirectcontributionstotheannotationprocess.Obviously,crowd-sourcingisoneofthosetrendsinpublichistorywherethehypecanoftenexceedthereality.However,thereareagrowingnumberofexamplesofwell-designedwikiorcrowd-sourcingendeavorsthatprovetheunderlyingvalueofsuchoutreach.“DecodingtheCivilWar”fromtheHuntington,forexample,offersanintriguingmodel(https://www.zooniverse.org/projects/zooniverse/decoding-the-civil-war).AttheHuntington,theyhavetakenacollectionof15,000encipheredtelegramsfromthe

Pinsker

-4-

ThomasEckertPapersandofferedthemupto“citizenarchivists”tohelpdecodethroughaZooniverseplatform.PALactuallyparticipatedinthiseffort(oratleastformerDirectorDanielStowellassistedinsomefashion),andtheyseemtobemakingfairlyrapidprogress,claiming50%completionafterjustoverayearofseriouspublicwork.Mystudentshavecertainlyenjoyedtheirexperiencewithit.

Duringtherecentsitevisit,PALstaffmadeclearthatonestumblingblocktotheirprogresswasasetofexpectationsforindexingandannotatingdocumentsthathadacomplexarrayofindividualorplacenames(suchaspetitions).Tome,however,thisseemslikeagoldenopportunityforcrowd-sourcing.PALstaffshouldidentifyallsuchencumbereddocumentsandpostthemtogetherina“wikiseries”thatmightuseZooniverseoracomparableplatformtoallowcrowd-sourcedannotationsorindexing.DoingsowouldnodoubtrequiresomeextrainitialworkbyPALstaff(orperhapsbyoutsideconsultants),butovertime,itwouldnotonlyreducemuchmonotonous,time-consuminglabor,butalso(andmoreimportantly),shouldhelpengageakeyaudiencevitaltotheultimatesuccessoftheproject–notjustgenealogistsandlocalhistoriansatlarge,butmostespeciallyIllinoisgenealogistsandlocalhistorians.Infact,IcanhardlyimagineanyinitiativesmoreattractivetoIllinoishistorybuffsthanapublicoutreachthatreallytriedtoengagethemas“ContributingEditors”toPAL.Thetestseriesrepresentstherealheartoftheproposedexperiments,however.Ifateachingseriesandawikiseriesseemalmostgimmicky,oratleastlimitedintheirscope,thenitismostdefinitelythetestserieswhichbreaksthroughtowardwhatthefounders(andfunders)ofPALoriginallyenvisioned.Butthekeydistinctionhereis“test”orwhatSiliconValleytypesalwayslabelas“beta.”IfPALcommitstoreleasingitsdocumentsonlineoncetheypassthroughcertaincriticalearlystagesofediting(transcription,proofing,markup,indexing),butwithoutregardtochronologicalcompletenessorevenwithoutfinalannotationsorreviewnecessarilycompleted,thentheycandramaticallyacceleratetheonlinepublicationschedule.Ofcourse,

Pinsker

-5-

thedangerofsuchanapproach,andthereasonwhyithasn’tbeenadoptedearlier,isbecausethereisalwaysafearinanyhistoricaldocumentaryprojectaboutreleasinginadequateworkproductandthendisseminatingpoor-qualitymaterials.However,atestseriesfullylabeledassuchanddisseminatedsimultaneouslywiththestartofafullycuratedteachingseriesandawell-designedcrowd-sourcedwikiseriesoffersinmyopinionacoherentandappealingpackage.Aneffectivetestseriesmustbeformattedtoencourageuserfeedback–bothinerror-reportingandthroughstandardonlineusersurveysabouttheinterfaceandsearchexperience.Theprimaryaudienceherewouldbeforseriousresearchers.InthePapersoftheFoundingFathersproject,theydescribedthisideaas“pre-publicationaccess”–anintenselyappealingframeworkforresearchersofallstripes.Alloftheideasaboveconcernonlinepublication–butwhataboutprint?IdoendorsetheideaoftryingtoissuewhatwouldrepresentineffectatwelfthvolumeintheCollectedWorks(CW)printseries–onethatfeaturesallofthemajordiscoveriesofPALsincethepublicationofthe1991CWSecondSupplement.Thatseemstobethemostobviousnextstepinprintpublication,butthereareotherprintvolumesworthconsidering,eitherinthenear-termorlong-termfutureoftheproject:

• LincolnNeverWrote(orSaid)That–StateHistorianThomasSchwartzusedtopensuchacolumnforthenewsletteroftheAbrahamLincolnAssociation.ItisanintriguingideaforthePALtoconsiderattemptingtomineitsfilesforeitheraslimprintorcompaniondigitalpublicationthatexplainswhysomeLincoln“documents”havebeendeterminedtobefalse,wronglyattributedtohim,orinotherwayssuspect.

• AttributedWords–Don&VirginiaFehrenbachereditedavolumeofLincoln’sRecollectedWordsinthe1990s,buttheirworkexcludedmanyCabinet-leveldirectivesthatclaimedtobequotingLincoln(“ThePresidentdirectsmetosay…”).ThesestatementsarecurrentlywithinthePALdefinitionofa“LincolnDocument,”butperhapsdeservetheirownprinteditionbecausetheyrepresentsuchaspecialcategoryofmaterial.Ifeditedproperly,suchavolumemightappealtoacademicscholarsasastand-alonepublicationjustliketheFehrenbacher’spopularedition.

Andultimately,Iwouldstillendorsetheideaofafinalcomprehensivepublicationscheduleorganizedaroundastandardchronologicalseries--asalmostallgreatdocumentaryprojectsinvariablydo,butonlyafterspendingatleastthenextfewyearsrollingoutthesevariousaudience-focuseddigitalseriesandlimitedprinteditionsasprecursors.

Pinsker

-6-

Recommendation#2:PALShouldModifyItsScopeofResearchandEditorialProceduresAprojectlikePALmustfirstandforemostserveanaudienceofscholarsandseriousresearcherswhoareseekingreliabletranscriptsofessentialhistoricaldocuments.ButwithafigurelikeLincoln,aprojectlikePALmustalsoproceedwiththeneedsofbroaderaudiencesinmind,becausethereisparticularinterestinLincolnfortheclassroom,politicalcircles,andamongthegeneralpublic.Withthatinmind,therearethreemaincategoriesofcurrentPALprioritiesthatseemtometobeworthmodifying:

• DefiningLincolnDocuments• UsingPageimages• AnnotationProcedures

DefiningLincolnDocumentsPALhasadoptedaveryexpansivedefinitionofa“Lincolndocument”thatincludesnotonlytraditionaldocuments(writtenbyALinhisownhandwritingoraddresseddirectlytoAL),butalsotheentiretyofallenclosureswithinhiscorrespondence(includingwholecourt-martialcaserecords,forinstance)andallpublishedstatelegislativeandcongressionalrecordsfromhispoliticalcareerintheIllinoisGeneralAssembly,Congress,oraspresident.Icannotidentifyanothermoderndocumentaryprojectthatincorporatessuchanexpansivedefinition.Itseemstobetrulyparalyzingasascopeofresearchandutterlydisconnectedtotherealneedsofitscoreacademic/researchaudience.Onekeyprincipleofgooddocumentaryprojectshasalwaysbeentoavoidreinventingthewheel,orinotherwords,nottoincludeinthescopeofwork,materialsthatareeasilyaccessibleinotherpublications.So,ifLincolnreceivedalegislativereportinacorrespondence,thereisnoneedtoincludeaversionofthatreportifitisalreadyavailableelsewhereinprintoronline.Thisseemsfundamental,butaccordingtoPALstaffobservations,itisaprinciplethathasbeenignoredinthisproject.Still,Iwouldnotrecommenddiscardinganyoftheworkalreadycompletedunderthisexpansivescope,butIwouldurgePALtoredefineenclosuresandlegislativedocumentsintoaclearsecondarytierofmaterials.Anythingfromthissecondtierthathasnotalreadybeentranscribed,markedupandannotatedshouldimmediatelybeputtotheendoftheselineoftasks–regardlessofwhereitfitsinthechronology.Moreover,ifmyotherrecommendationfora“wikiseries”iseverattempted,thenIwouldsuggestthatannotatingsuchsecond-tiermaterialsisexactlythekindofarduousbutperipherallaborthatshrewdcrowd-sourcingcanhelpaddress.UsingPageImagesOneofthemostimportantfacetsofonlinepublication,especiallyinanydatabaseformat,istotrytoofferthepresentationofpageimagesalongsidetranscriptions.Thisfeature,especiallywhencoupledwithenhancedsearchfunctionality,issomethingthatprintpublicationssimply

Pinsker

-7-

cannotduplicateandwhichseriousresearcherinvariablyappreciate.YetthisobjectivehasnotreceivednearlyenoughattentioninthecurrentPALscopeofresearch.First,andmostimportant,PALmustprioritizesecuringthepermissionsnecessarytoshowpageimagesatleastforallofthetoppriorityLincolndocuments.Iwassomewhatshockedtodiscoverthatthisbasicstepintheresearchprocesshadnotbeenundertakeninanysystematicwayandwasonlynowatthislatestagebeingorganizedincoherentfashion.IreallycannotthinkofanythingmoreessentialtosuccessfulonlinepublicationthanbeingabletodemonstratehighresolutionimagesofLincoln’sdocumentsthatarealsofullysearchable,expertlytranscribed,fullyannotatedandeasytoreadacrossmultipledevices.ThisiswhatwillearnPALareputationasoneofthecountry’spremierdigitalhistoricalprojects.Itmustreceiveabsolutepriority.AnnotationandEditorialProceduresAmongthecurrentPALstaff,thereseemstobedeepconcernoverinconsistenciesineditorialproceduresthathavebeenallowedtodevelopovertheyears.Inanylarge,complex,multi-yearproject,thisisboundtooccur,butitisespeciallycriticalatthisstagethatPALconductsathoroughreviewofitsprocedures,straightensoutanymajorinconsistencies,simplifiesitsprocessasmuchaspossibleandputsallofthestandardsintowriting.Thereshouldbeupdatedandeasy-to-followmanualsavailablefortrainingstaff,aswellasforeducatingvolunteersandconsultants.Inparticular,PALstaffhasidentifiedseveralpointsofcontroversyorconfusionintheircurrenteditorialproceduresforannotations.Allofthemseemtobequitetypicalproblemsforlarge,complicatedprojects–especiallytheargumentsoverhowtolimitextraneoushistoricalinformationorhowtoavoidexcessivebiographicalorgeographicalcontext.Documentaryeditorsneedmodestannotationgoals,butthegoodonesarealwaysblessed(orcursed)withsomuchbackgroundknowledgethattheyhaveadifficulttimecuttingthemselves(oreachother)offfromtheinevitabledetours.Idon’treallyknowhowtoadviseotherthantoreiteratewhateverybodyknowstobetrue–justkeepannotationsshortandsimple.Mybestadviceistoimposeahardwordcountcombinedwithgeneralguidanceandotherwisejusttrytoleteditorsrelyontheirbestdiscretionwithinthoselimitations.

Pinsker

-8-

Recommendation#3:PALShouldUseDigitalToolstoIntegrateEarlierProjectsOneofthekeychallengesfacingPALastheprojectcontemplatespublicationishowbesttointegrateearlierandvaluableprojectsliketheLincolnLegalPapersorTheLincolnLog.Therearesomedigitalmashupsthatoffercreativewaystodoso.ProgrammerscancombinerecordsfromPAL,LLP,andLLinacombinationtimelineandmapsothatresearcherscanengageinakindofone-stopsearchexpeditionforrelatedLincolnrecords.WemodeledthisconceptfortheJournalofAmericanHistoryasearlyas2009inour“BuildingtheDigitalLincoln”exhibit:http://archive.oah.org/special-issues/lincoln/media/pinsker/documents_artifacts/timemap.html

Allthreetypesofrecordscouldbefeaturedhere

ClickinglinkswouldtriggertextboxwithdirectrecordaccessandlinkstorelatedLincolndocuments

Userscanscrollontimelineaboveorbymapbelow

Pinsker

-9-

Recommendation#4:PALShouldAdaptItsDigitalPlatformEverybodyinvolvedinPALacknowledgesthatthedigitalinfrastructureoftheprojectiscomplicatedtosaytheleast.ThereisPubMan,aproprietaryandweb-basedCMSnowlicensedbyacompanyknownasIDM.PubManismostdefinitely“older”digitaltechnology,butthisparticularCMShasareasonablyimpressivepedigree–projectssuchastheMillerCenteratUVA,JeffersonPapersatPrincetonandCambridgeDictionariesatCUPhaveallutilizedthisplatform.ThePubMansystematPALnowcontainsover150,000documentswithtensofthousandsofmoretocome.Digitalarchiving,however,isyetanothermatterandinvolvesathird-partyfirm(BlueMountain)andalsotheuseofsomereservedserverspaceatGettysburgCollege.Moreover,littleaboutthecurrentstructureofthePALPubManplatformseemsdesignedforeaseofpublicuse.Theinterfaceisdifficulttomasterandqueriesarenotintuitive.Asbothascholarandnowmostrecentlyasaconsultant,Ihavebeensurprisedathowdifficultitistousethesystem.Thereareothermorepopularandopen-sourceCMSoptions,suchasDrupal,thatmightprovidelivelierend-stagepublicationplatformsforPAL.AttheHouseDividedProject,weuseacombinationofDrupalandWordpresstoengageourprimaryaudience,whichisK-12educatorsandtheirstudents.Iheartilyrecommendthesesystemsasbotheasytouseandrobustintheircapabilities.However,therearesomeobviousincentivesforkeepingPubManinplace,especiallywithaspirationsforacceleratingadigitalpublicationscheduleandthusavoidingtoomuchre-trainingofstaff.IamnotaCMSexpert,butthereareundoubtedlywaystooverlayothertypesofdigitalinterfacesontopofPubMan,thatwouldmaintaintheexistinginfrastructurewhilestillcreatingafriendlierpublicfacefortheproject.OnehelpfulmodelforsuchaninterfacecomesfromtheDigitalPublicLibraryofAmericawhichallowsusersarobustsearchplatformcombinedwithincrediblyuser-friendly(andevenidiotprofessor-proof)browsingtools,suchastimelineslidersorclickablemaps.

Pinsker

-10-

Recommendation#5:PALShouldStreamlineStaff,ButExpandExternalContactsTherecentstatebudgetcrisisandstaffingturmoilatPALhascertainlybeendifficultforallinvolved,butsomeoftheworkplacechallengesattheinstitutionseemtorunevendeeper.Thecomplexorganizationalchartcreatedbytheuniquesetofstate/federal/universitypartnershipsthathaveguidedtheprojecthasproventobedeepsourceofdissatisfactionforseveralstaffers.Theyfeelneglectedandundervalued,ineffect,neitherherenorthereinthebureaucraticmazethatenvelopstheirwork.Thedualstaffstructuredoesnothelpthissensationeither.Butagain,myadvicereallyisjustcommonsense:theprojectneedsaclearerchainofcommandandabettersenseofcollaborationwithitskeypartnersinSpringfield.Perhapslessimportant,butstillvitalforthelong-termviabilityoftheproject,thereisarealopportunityrightnowtoexpandtheproject’snetworkofexternaladvisors.ThereisalreadyaneditorialandadvisoryboardfilledwithleadingscholarsandkeyfixturesintheLincolncommunity,butitseemsclearthattheirrolehassofarbeenrelativelymarginal.Itistimetotrytore-engagethoseadvisorsinprojectsthatmighthelppromotePALandshapeitspublicationformat.IsuggestedearlierthatofferingsuchadvisorsanopportunitytocreateexhibitsforaTeachingSeriesorPrimaryDocumentsSetmightbeaneffectivewaytomoveforward.Suchexhibitscouldalsobeusedasawaytorecruitnew,moreactiveadvisors.PALshouldalsoconsideraddingaTeachersBoardandaDigitalAdvisorsBoardtofurtherenhanceitscurrentadvisoryframework.

ItisalsotimeforPALtoconsiderwhetheritcansupportasteam-linedDC-basedstaffingoperationwithacademicpartnerships.ThereareprogramsandscholarsintheregionwhomightbeenticedtoorganizeteamsofstudentresearcherstoassisttheprincipalDCstaffinitsefforts.ThecostoftheoperationsinDChasbeenhigh–byoneestimate,PALhasspentapproximately$3.5millioninordertoconductrecordssearchesattheLibraryofCongressandtheNationalArchiveswithpresumablymillionsmoretospendunderthecurrentscopeofresearchparameters.EvenatDickinsonCollege,whichhasnograduatestudents,wehavebeenabletoorganizeasuccessful,grant-fundedefforttocreateadigitalarchiveofrecordscopiedfrommaterialsattheNationalArchivesthatrelatetotheCarlisleIndianSchool(http://carlisleindian.dickinson.edu/).LargeruniversityprogramsshouldbeabletodoevenmoreforPAL–especiallywithgoodstaffsupervision—thatwouldhelpacceleratethetimetableforcompletionandultimatelysavetheprojectmoney.

Anotherwaytocultivatedeepercontentpartnershipswithacademicsandtheirinstitutions,mightbetoinvestinshort-term“fellowships,”whereselectedscholarswouldbereimbursedfortraveltoSpringfieldtoassistPALstaffinpublication-relatedtasksonashort-termbasis.Theabilitytobringinafreshsetofoutsidebutstillexperteyesmightbeworththeassociatedcosts,especiallygiventhecurrentreducedstateofthepermanentPALstaff.Moreover,ifPALcouldsomehowintegrateasmall-scalefellowshipprogramwithstandingacademicpartnerships,thentheymighthavethebestofbothworlds–seriousbutlow-coststaffingsupportcombinedwithhigh-levelprojectpromotionanduserfeedback.

Pinsker

-11-

Conclusion

OneofthedefiningmomentsofAbrahamLincoln’scareercameinhisbehind-the-scenesbattleswithGeneralGeorgeMcClellan.Fromlate1861untillate1862,Lincolnrepeatedlypushedhisstubborncommandertobemoreaggressiveinthefield.McClellanwasanexceptionaltalentinmanyways,butduringthefirsthalfoftheCivilWar,heclearlyallowedperfecttobecometheenemyofgood,astheexpressiongoes.Atonepoint,onApril9,1862,Lincolnwrote:

IbegtoassureyouthatIhaveneverwrittenyou,orspokentoyou,ingreaterkindnessoffeelingthannow,norwithafullerpurposetosustainyou,sofarasinmymostanxiousjudgment,Iconsistentlycan.Butyoumustact.

Youmustact.ThePALisstaringdownatitsownGeneralMcClellanmoment.Theprojecthasdoneexcellentworkovermanyyears,butlikemanyothersuchsprawlingdocumentaryefforts,itissaggingsomewhatundertheweightofitsownperfectionismandcomplexity.Lincolnhadmanyfaultsasanadministratortogoalongwithhisextraordinarytalentsasacommunicator,butonethinghegotrightwithMcClellanwasthatsometimesyoujusthavetoact.Mostofmysuggestionsinthisbriefreportcomefromthatspirit.Bold,persistentexperimentationiswhatisneededrightnowasPALmovestofindwaystoaccelerateitspublicationschedule.Thechallengeisadauntingone.Butultimately,thegoodworkalreadyaccomplished,andtheinspiringexampleofthegreatpresidenthimself,willsurelycarryPALthroughtofinalvictory.

Report—The Papers of Abraham Lincoln

Jennifer Stertzer

1 April 2017

Introduction

The Papers of Abraham Lincoln is a preeminent scholarly editing project, long admired in the

community for its breadth and comprehensiveness, as well as its willingness to engage with a

complex online content management system. Though the project has faced organizational

challenges over time, and significant management issues during the past year, I am confident

this project can be successful. The following report seeks to: 1) briefly review the project’s

history and current state, focusing on the challenges/weaknesses and successes/strengths, and;

2) identify next best steps, both in the near term, or “interim period,” and in the long term.

History and Current State of Project

The conversations that took place during my visit were intense and informative. I see no reason

to revisit much of what was discussed, except to note a few weaknesses/challenges and

strengths/successes that are useful in evaluating the current state of the project and

developing next steps.

Weaknesses/challenges:

● uncertainty of past director’s future involvement

● lack of project mission

● complicated and incomplete workflow

● did not collect permissions for image use*

● lack of definitive scope

● significant collection challenges

● no publications to date

Successes/strengths:

● dedicated, experienced staff

● supportive administration dedicated to getting the project back on track

● despite recent challenges, the project is one of the premier documentary editing

projects in the field

● an enormous and diverse user base that will be thrilled to use the digital edition

● a significant collection of materials in various stages of the editorial process

*I note here that this issue is currently being addressed. While I do not cover this issue in the

report, I think it wise that you decided to tackle this task now.

Project’s Future—Interim Period

I will start by acknowledging three issues that have the potential to complicate the project’s

interim strategy: 1) still waiting to learn the final decision regarding the past director; 2) the

immediate need for reconceptualization and reorganization, and; 3) determining how involved

a new director will be in reimagining the project. In my opinion, the immediate needs of the

project take precedence. While it would be ideal for the new director to lead and participate in

this process, the current situation unfortunately does not allow for this. Furthermore,

continuing with the current workflow and lack of direction will only further delay project

success and demoralize the staff. This interim period is critically important and steps can be

taken during this time that will address immediate needs and make it easier for the next

director to continue the revitalization in creative ways and build upon the new foundation. I

have broken up my recommendations for next steps into several categories. Some suggestions

are minor, such as those related to the website, while others will require an investment of time

and possibly resources. Taken together, these suggestions were developed to achieve two

goals:

1. Define the project, both in terms of its mission as well as its external relationships.

2. Develop a sustainable, manageable editorial methodology and workflow for the project

that results in publication.

Project personnel. I was very impressed with the staff’s knowledge, dedication, and

enthusiasm for the future. Staff involvement during this interim period is critical and the

success of this process depends on their engagement, feedback, and expertise. On a side note,

I did sense a divide between the Springfield and DC staff, a divide that was also very apparent in

the “ideas for the future of the project” statement they each prepared. I don’t know enough

about the project’s history, politics, or individual personalities to speculate as to why this divide

exists. However, I think it’s important to rethink position descriptions during the project

reconceptualization and provide additional opportunities for the DC staff to contribute to the

project. I imagine searching for and scanning documents is tedious work. Furthermore, it’s a

waste of their talents and is a disservice to them; it will be difficult to find work at another

documentary editing project if all they’ve ever done is search and scan. Reconceptualizing the

project, developing a clear work plan, and migrating to a more appropriate platform should

allow the DC staff to be engaged with the project in several meaningful and productive ways.

Relationship with UIS. This relationship needs to be discussed and either defined or

abandoned during the interim period. If possible, I recommend Alan and Sam meet with UIS,

explain that the project is being reconceptualized and that the relationship needs to be

determined. Before that meeting, think about the benefits for both the project and the

university. What are the advantages of this relationship? Do they or can they provide any

support beyond salaries? Make a list of the benefits this relationship could or does provide,

such as: 1) salary and fringe benefits; 2) ability to use these salaries for cost sharing on federal

grants; 3) institutional host can be mentioned in grant applications, in promotional materials,

and online; 3) possibility for dual appointment, such as assistant editor and assistant

professor—academic rank can be helpful when editors submit articles for publication or

present at academic conferences; 4) IT support, hosting of either digital edition or archived

images. Next, think about what benefits the university will realize from a relationship with the

project. Let them know that PAL has the support of the ALPLF/ALPLM, is reconceptualizing to

reestablish itself as a premier documentary editing project and THE source for Lincoln, and is

working on a plan to build an innovative and robust digital edition. Additionally, there are

opportunities for PAL staff to work with students, teaching them editing, research, and digital

skills. UIS can publicize this partnership—universities like to take some credit when projects

receive federal grant funding or are recognized for scholarly contributions. This is an

opportunity for them to make a commitment to support this project and in turn share some of

the attention that will come the project’s way.

Relationship with ALPLF/ALPLM. Based on some of the staff feedback, PAL’s relationship with

the ALPLF/ALPLM needs to be better defined and all staff (PAL, ALPLF/ALPLM) need to be made

aware of the relationship’s specifics and expectations. What is expected of the PAL editors?

Are they responsible for answering questions, performing research, and/or dedicating time to

help the ALPLF/ALPLM? Can the PAL staff participate in ALPLF/ALPLM activities, planning

committees, educational programs, etc.? How can the PAL leverage this relationship to

facilitate fundraising and increased publicity? How can the PAL’s expertise and experience be

drawn on by the ALPLF/ALPLM? Both the PAL and ALPLF/ALPLM are in a position to benefit

from this relationship.

PAL Website. I recommend updating the site, perhaps adding a new story to the “latest news”

page that will in turn replace the news story that’s currently on the homepage (dating back to

2015). Also update the copyright information in the site’s footer. Keeping this site fresh

indicates the project is still alive and thriving! I see the PAL’s Facebook account is somewhat

active—that’s great! If it’s helpful, the ADE can share the PAL’s posts. Additionally, we can add

PAL to the list of future featured projects.

Pubman. It’s obvious that there was a significant investment of time and resources into this

system and I commend the staff for their work using this CMS effectively and extensively.

While the staff have devised a Pubman workflow, making use of Pubman’s features and

structure has burdened the project—excessive amounts of time spent tagging, checking

documents in and out of the system in order to edit, no way to control permissions at a

granular level, no easy way to query collection. There are better systems to manage both the

scope and of this project and variety of content to be included in the eventual publication. As

such, I strongly suggest you consider moving to a new content management system and

publication platform. While I have ideas for appropriate solutions, I think it prudent to go

through the traditional process of choosing a platform. This can be done while

reconceptualizing the project as a whole, addressed below.

Reconceptualizing the project. Thinking in terms of developing a strong foundation and a

manageable, sustainable project-level work plan, I suggest organizing a training / project

rebuilding workshop. This workshop could be loosely modeled on the Institute for the Editing

of Historical Documents (Camp Edit), which covers all steps required in defining, organizing, and

managing a project as well as editing, annotating, indexing, and selecting project-appropriate

tools and platforms. This workshop should involve all PAL staff, who possess a deep

understanding of the project and content, and ideally two outside participants with

backgrounds in project conceptualization, best practices, management, implementation, digital

editing, and publication. The two outside participants would be responsible for planning,

organizing, and leading the discussions and planning activities. The PAL staff would need to be

prepared to talk about all aspects of the project, participate in the development of an action

plan, and commit to its implementation. Below, I have outlined and detailed topics that need

to be addressed during the workshop. The topics to be addressed include:

1. Developing project mission

2. Identifying content, platform/CMS, and publication goals

3. Rethinking organization and control

4. Redeveloping a plan for collection

5. Selecting and organizing documents for publication

6. Rethinking transcription

7. Indexing, searching, and tagging

8. Rethinking annotation

9. Choosing a platform

10. Designing a digital edition

Developing project mission and goals. There are two critical steps projects must take in the

beginning: developing a mission statement and conceptualizing the edition. All subsequent

decisions, from determining scope and organization to developing workflows and identifying a

platform, will be guided, determined, and influenced by these first two steps. The mission

statement should answer three questions: 1) key market—target audience(s)?; 2)

contribution—what will the project provide?, and; 3) distinction—what makes your project

unique? On the current PAL site, there is the beginnings of a mission statement: “The Papers of

Abraham Lincoln is a long-term project dedicated to identifying, imaging, transcribing,

annotating, and publishing all documents written by or to Abraham Lincoln during his entire

lifetime (1809-1865).” The orientation materials handout I received also hints at a mission: “...

the Papers of Abraham Lincoln (PAL), a documentary editing project dedicated to locating,

imaging, transcribing, and annotating all documents written by or to Abraham Lincoln during

his lifetime. PAL’s mission is to provide a free, digital, comprehensive edition of the entire body

of Lincoln’s writings, including incoming correspondence.” Obviously, these two statements are

similar but also suggest two vastly different scopes. Not having a clear sense of what type of

project this is—comprehensive (and that’s open to various levels of interpretation), selective,

hybrid—has resulted in the current state of the project—lots of work but no publication of

materials. These statements provide a good starting point to begin developing a clear mission.

This mission statement will then inform all other project decisions. Identifying project specific

goals will help the project conceptualize the mission and edition. There are three areas the

team will want to address:

1. Content-specific goals:

a. Scope / comprehensiveness*

b. Metadata

c. Transcription and data

d. Annotation, taxonomies, glossary

e. Internal Linking

f. Indexing

2. Platform-specific goals:

a. Open-source

b. Flexible editorial / publishing tools

c. Ability to download data

d. Powerful search and browse functions

3. Publication goals:

a. Digital

b. Print

c. Both?

*The project’s scope will require a significant amount of work. Based on current estimates, the

collection work requires another 10 years at current staffing levels. Considering this is just to

search for and scan the documents, the time estimate for preparing edited publications of

these materials will be considerable and most likely unfeasible. A clear mission will help

define/narrow the scope but additional work will be necessary. The team needs to consider

how to deal with what are essentially ancillary/supplemental materials. Should these materials

be included? And if so, how can a hierarchy of materials be established so that some

documents are prioritized and receive comprehensive editorial treatment and other

ancillary/supplemental materials are only catalogued, used in annotations, linked to, associated

with metadata, etc.? Figuring this out is critical.

Organization and Control. At present, documents are housed in Pubman making use of that

system’s framework of document organizational structure: documents, titles, and packages.

This framework presents a very flat picture of what the collection contains and it seems very

difficult to query by date, author, recipient, title, etc. With a collection this large, editors need

to be able to quickly navigate through materials and perform complex queries. Editors also

need to be able to apply varying methodologies within the system—not all documents require

the same amount of editorial work. When developing a system for organization and control,

there are several goals: 1) physical and intellectual control; 2) easily search and browse

materials; 3) generate reports, and; 4) manage workflow. The current system falls short on/or

doesn’t meet these goals. That said, a new system can make use of parts of the old system.

Collection. When developing a collection policy, projects define what they are seeking (based

on scope), carefully look and records what they find and we they don’t find, and then process

the new documents, making use of the carefully developed organization/control system. The

PAL’s lack of a definitive scope has significantly expanded the collection process and this will

need to be addressed.

Selection and organization. The selection and organization of documents to be published will

be informed by the project’s scope and mission: what documents should be considered, how

should they be evaluated, and how to choose between documents that cover similar topics. In

designing the selection policy, first tackle large-scale issues:

1. How inclusive will the edition be? Selective, comprehensive?

2. Will the edition be based on a topic, theme, or AL’s life?

3. Will the edition be limited to correspondence, diaries, speeches, articles, or other

formats? If the edition is thematic, what boundaries will there be on selection? Limit by

archive, chronology, or types of objects?

4. Will it be based on a single archival collection or document (such as a diary), or will

objects from multiple repositories be included?

5. If editing correspondence, will letters both to and from the subject be included? What

about third party correspondence?

6. Does the edition have start and stop dates?

7. What is the desired number of documents? Will the edition be bound by resource

limitations, time limits, or space?

8. Will portions or excerpts of documents, whole documents, or both, be included?

Once parameters for the materials to include in the edition has been determined, evaluate the

content of the documents: determine the themes to cover and rate the documents accordingly;

determine whether a representative sample of various topics covered in the archive should be

included; identify well-known documents/correspondents that need to be included, and;

determine whether only unpublished materials will be included. After the team has identified

documents to consider for possible inclusion and evaluated the types of content contained

within, a selection policy that reflects the decisions made can be designed.

Organization options in print editions are limited—tables of content lay out the overarching

organizational model for the edition, which is primarily chronological, followed by

transcriptions and annotations, and finally an index. But how to organize a digital edition? Are

tables of content necessary or just carry overs from the world of print? There are a variety of

options for organizing objects into a digital edition. Divisions, or rationales for grouping

objects, can result organically from decisions made during the selection process. Organizing

objects thematically or chronologically, or dividing based on format (document type, version) or

location (archive, repository) are all perfectly acceptable ways to shape an edition making it

possible for the user to choose any of these options to approach the edition. But it is helpful to

remember that this arrangement will likely be a user's initial experience; digital tools and

platforms can allow users to search, browse, and assemble materials in ways that suit their

interests. So while it is important to carefully craft the default layout, also providing options for

alternative navigation and assemblage will ensure the edition is broadly accessible.

Next, think about the ways in which users will want to access the content: chronologically, by

author/recipient, title, format, index/taxonomy terms, repository/collection, location,

document type, and/or version. Creating digital editions that can be explored systematically

and extensively like this requires structured metadata. Metadata, or data that describes other

data, is essential when working with diverse materials. Abbreviations, shorthand, inconsistent

spellings and terminology can be made reliably searchable and browsable if the metadata is

both congruous and thorough. The process of managing metadata is similar to that of

cataloging objects—create a system to identify content that will benefit from metadata,

develop a controlled vocabulary, and consistently apply.

Transcription. Transcription is the backbone of a documentary edition. It makes difficult

primary sources legible to the general reader and enables editors to publish these sources and

make them widely available. In a digital edition, transcription becomes even more important

because it serves as the basis for searches and interpretative tools such as textual analysis.

Editors have developed unique and descriptive ways of editing and publishing complex primary

sources in print format to try to represent the original. Digital publication platforms, however,

allow for easy incorporation of images of the source materials and encoding and database

solutions make it possible to display multiple transcription views. These options could

potentially affect the way editors approach transcription—is it necessary to spend lots of time

formatting documents when an image can be presented alongside the transcription? When

considering how faithful to be to the original text, it is helpful to keep in mind the intended

audience, design of the edition, and search and display plans. Document type should also

influence this decision, as authorial spacing and formatting might be important to capture (e.g.

financial documents, reports, and accounts). Ultimately, editors will always struggle with the

balancing act of making documents accessible and making transcriptions accurate

representations of the original. Regardless of the chosen method, it is of utmost importance to

develop guidelines and rules, rigorously follow them, and make available to users.

The PAL follows a general policy of diplomatic transcription, which is fairly standard for the

field. PAL also plans to offer a clear text transcription and are using XML to facilitate. While

making use of clear text is appropriate in certain situations, I would advise reconsidering this

approach. What is the rationale for this decision? Are the benefits worth the extra work? Is it

necessary if you are also providing an image of the manuscript? Also, decisions to standardize

elements of a document only require a mention in the statement of editorial methodology.

Indexing. I realize the PAL is most likely not thinking about an index but bear with me here.

Thinking about indexing and how to structure information is an important exercise and key to

developing robust digital editions. Considering the types of content currently being annotated

by the PAL, making use of an expanded index—I’m thinking here of descriptive

taxonomies—could alleviate pressure from traditional annotation. Instead, the PAL could

refocus traditional annotation efforts on contextualization. That would leave the work of

developing and linking IDs—there are other platforms that will make this work more efficient

and result in a better publication.

Indexes help make content accessible and understandable. While it might seem like a relic of

the print edition, thinking in terms of an index’s hierarchical structure and its economy of words

dovetails nicely with building effective taxonomy lists and tagging schemas. An index is a

systematic arrangement of entries designed to enable users to locate information within an

edition. Structurally hierarchical, the top-level entry headings list people, places, events, and

major subjects within an edition, usually presented alphabetically and subdivided into logical

subentries. An index should be considerably more than an outline and considerably less than a

concordance of words and phrases. In short, a good index makes the content accessible. As

with annotation and tagging, it is helpful to keep several things in mind when developing

indexing guidelines. First, entries and subentries should reflect the nature of the materials in

the edition and should be phrased to serve the needs of the edition's audience. Editions of

financial, legal, scientific, or other specialized topic might use technical phrasing. Also think

about what types of questions users will ask of the material's contents—who, what, when?—

and consider what terms users will intuitively search. Also think of ways to point users in

directions they might not have considered. Finally, when an edition focuses on a single person,

organization, topic, or event, entries under that heading need to be broken down into

intelligible subentries. Don't just make a list of references; use descriptive subentries to help

users easily find information.

It is also incredibly helpful to keep the index's design in mind during the editorial process. Start

with what subjects, places, themes, and people are important in the edition, and develop and

manage a list, preferably in the content management system/database. Next, consider how the

content is related and where cross-references be effectively used. Making connections between

related documents, subjects, people, and places is enormously helpful to the user. Within the

platform, maintain a preliminary annotated index linking mentions with entry. Keeping track of

where a person is mentioned and associating that reference with a well-formed index entry will

help avoid confusion in the end—which John Smith is it? As with other steps in the editorial

workflow, develop a statement of method and make it available to users, explaining how and

what has been indexed. And last, but certainly not least, be consistent! Planning is key and

worth every minute spent.

Annotation. Editors have a variety of techniques with which to make objects intellectually

accessible. In addition to capturing metadata editors can supply annotations, tag content, and

develop indexes and taxonomy lists. In the context of digital editions, annotations are broadly

conceived: introductions, headnotes, source notes, endnotes and footnotes, biographical and

geographical directories, timelines and chronologies, maps and gazetteers, family trees,

photographs and illustrations, glossaries, visualizations, essays, metadata, and taxonomy lists.

All types, however, should serve the same purpose: annotation is added to make materials

more understandable, provide context, and add value. When designing annotation, keep in

mind the nature of the materials and the needs/interests of the audience. There are several

elements to consider when developing annotation guidelines: first, who will use the

edition—scholars, students, the general public; second, what are the objects like—do they

detail events or topics that might be unfamiliar to most people, do they cover topics that are

not commonly known, and do they require specialized knowledge to be understood; third, in

what context will they be read, as a standalone resource or used in conjunction with other

resources; and finally, how selective is the edition—a highly selective edition needs to provide

more context, alerting users to and/or summarizing documents not included in edition as well

as pointing to related materials.

Traditionally, editions provided readers with enough annotation to understand the texts:

people, places, organizations, and events were identified; summaries of preceding and

proceeding histories added, and; source notes containing additional information about the

original manuscript followed its transcription. A digital edition should include this type of

annotation; digital platforms make it easier, though, to repurpose and reuse this content.

Common digital tools and platforms—XML, Drupal, Omeka, WordPress—offer a variety of

technical strategies and options to integrate stand-alone as well as shared annotations. The

documents, content, annotation policy, and desired search/browse/query features will help

shape these decisions. For example, events described in the documents might need

stand-alone annotation in order to provide specific contextualization. However, the editor

might also want to create an ID. for that event so that every time the “it” is mentioned in a

casual way, a user can refer to a single explication. In addition to decisions regarding what and

how to annotate, editors will also need to decide where and how annotations will display.

Digital tools and platforms currently in use provide a variety of display options:

document-centered notes that stay with the document, sidebars, links, pop-ups, hierarchical

taxonomy lists, etcetera. Display decisions will also be made considering the same questions

listed above in conjunction with the capabilities of the digital tools and platforms in use.

The current annotation policy is unwieldy and complicated by the use of Pubman [annotation,

broadly conceived, includes more than just traditional annotation, as explained above]; I have

heard stories from other projects that get so wrapped up in work of tagging documents,

deciding/debating which tag to use, slipping into a rut of tagging everything that the point of

annotation and tagging is lost. It’s obvious the staff spend a lot of time engaged in this work.

Despite a detailed and long annotation policy, absent is a clear understanding of purpose. This

is not a critique of the staff; developing a clear, efficient, appropriate annotation policy requires

a defined mission, established goals (most importantly scope!), set publication plan, etc.

Unfortunately, previous annotation work might not have a place in the publication but that’s

not to say these things will be discarded. Refining the annotation policy should also include

thinking how best to repurpose content that will not be used for annotation—there are lots of

exciting possibilities (blog, social media, articles) that can benefit the project and its mission.

Choosing a platform. For PAL, moving to a platform that facilitates all aspects of the editorial

process as well as publication is critical. Once decisions have been made in the other

categories, you can start thinking about the best platform for the project; I always recommend

that technology choices be content and process-driven when possible. In general, a platform

should:

● be able to manage collection, both in terms of organization and control as well as the

project’s scope (size of collection);

● facilitate editorial work (transcription, annotation, indexing, etc.) and manage workflow

(ability to assign permissions and generate assignments; provide multistep process that

aligns with project workflow);

● allow for cataloging and processing of newly collected documents;

● be able to handle different multiple document types, both in terms of manuscript type

(letter, report) and document hierarchy (primary, supplemental, ancillary) as well as

different formats (images, illustrations, maps);

● allow for a variety of metadata, taxonomies, and annotations;

● allow for on-demand publication of finished documents, and;

● be capable of powerful search, browse, display, and query functionalities.

Additionally it necessary to consider: 1) who will host this platform; 2) whether you have access

to technical support; 3) cost—just because something is open source doesn’t mean you won’t

need to spend money (server space, programming, design); 4) does the platform have an active

user community, and; 5) how much customization will need to be done.

Designing a digital edition. Once you’ve thought about and made decisions concerning all

aspects of the project, you can begin to design the edition. At the beginning of this process, it’s

helpful to look at other document-based sites to see what you like and what you don’t. Think

about the kinds of things that you want to provide. Make a list and mark items by priority:

must haves, like-to-have, and in-a-perfectly-funded-world; don’t forget that you will actually

have to create and maintain these features! Once this list is developed, an elaborate planning

process follows that requires engagement of all staff and results in a wireframe and site

specifications. If you would like a packet of materials to help with this process, please let me

know.

Project’s Future—Long Term Ideas

New director. The concerns and frustrations voiced by the current staff provide a roadmap of

the types of skills the new director will need. Based on their feedback as well as my assessment

of the organizational, editorial, and technical challenges facing the project, I suggest keeping an

open mind in hiring the next director. I don’t think it’s necessary for this person to be a Lincoln

scholar; instead, I think looking for someone with a background in managing documentary

editing projects will better serve the project, especially at this point in time. Good project

managers will realize their limitations and work to educate themselves and build partnerships,

collaborations, and effective boards to advise on matters beyond their areas of expertise.

Ultimately, there is no substitute for good management—it is so vitally important, especially as

you work to rebuild this project and trust among staff.

So what skills should this new director bring to the project? It’s a long list! I found Daniel and

Kelley’s views on how the next director should manage, as well as their ideas in general, very

helpful and I would take those ideas seriously. Taking their suggestions and adding a few of my

own, the new director should:

● Be responsible for establishing (or continuing) and directing workflow, including

developing comprehensive job descriptions for the staff;

● Have a strong understanding of the project’s editorial methodology, be familiar with

best practices, lead efficient project discussions concerning methodology that result in

policy, and have the ability to make decisions and follow through on the decisions;

● Respect staff’s opinions and experience, provide opportunities for growth and

development (training, conference participation, publication opportunities), and serve

as a mentor;

● Be responsible for maintaining current partnerships and work to build other

partnerships and collaborations;

● Be responsible for preparing grant applications and working on additional development

opportunities for the project;

● Be engaged in the field of documentary editing, be active in the ADE, and encourage the

staff to participate as well

Engagement with the field. Encourage the PAL staff to participate in the ADE and field-related

events, conferences, publications, and discussions. While the tendency is for the project

director to be the face of the project, this doesn’t need to be the case. In fact, it’s not even

ideal. Participation exposes editors to different points of view and innovative ideas, as well as

provides an opportunity for staff members to share their expertise and publicize the project.

The way I think about, when my staff engage in the community , make presentations, and

participate in discussion, the more successful we are as a project.

Professional advancement. I sensed that several staff were unhappy with their salaries and

paths for advancement. Realizing that there are funding limitations and that

advancement/promotion is almost always tied to money, I nevertheless want to make the case

for its importance. Staff need to be given a clear career path and they need to know that

consistently strong work will result in recognition. There are obviously lots of ways to make this

happen, whether through promotion, pay raise, or change in job responsibilities. The last is the

easiest fix—managers should train and offer opportunities that prepare staff for their next role.

With my team, I like to think that I am preparing them to take my job. Even if raises and

advancement aren’t possible in the next several months/years, still develop a plan and share

with the staff. Transparency in these matters is essential.