front page - european commission : cordis :...

PROJECT PERIODIC REPORT

Grant Agreement number: FP7-ICT-2007-1-215847

Project acronym: EU-ADR

Project title: Exploring and Understanding Adverse Drug Reactions by Integrative Mining of Clinical Records and Biomedical Knowledge

Funding Scheme: Collaborative Project – Small or medium-scale focused research project

Date of latest version of Annex I against which the assessment will be made: 25/05/2011

Periodic report: 1st □ 2nd □ 3rd 4th □ Period covered: from 01/02/2010 to 31/07/2011

Name, title and organisation of the scientific representative of the project's coordinator: Prof.dr. Johan van der Lei, Erasmus Universitair Medisch Centrum Rotterdam

Tel: +31 10 704 3050

Fax: +31 10 704 4722

E-mail: [email protected]

Project website address:www.euadr-project.org

http://www.euadr-project.org/

Declaration by the scientific representative of the project coordinator

I, as scientific representative of the coordinator of this project and in line with the obligations as stated in Article II.2.3 of the Grant Agreement declare that:

The attached periodic report represents an accurate description of the work carried out in this project for this reporting period;

The project (tick as appropriate) 1: has fully achieved its objectives and technical goals for the period;

□ has achieved most of its objectives and technical goals for the period with relatively minor deviations.

□ has failed to achieve critical objectives and/or is not at all on schedule.

The public website, if applicable

is up to date

□ is not up to date

To my best knowledge, the financial statements which are being submitted as part of this report are in line with the actual work carried out and are consistent with the report on the resources used for the project (section 3.4) and if applicable with the certificate on financial statement.

All beneficiaries, in particular non-profit public bodies, secondary and higher education establishments, research organisations and SMEs, have declared to have verified their legal status. Any changes have been reported under section 3.2.3 (Project Management) in accordance with Article II.3.f of the Grant Agreement.

Name of scientific representative of the Coordinator: Johan van der Lei

Date: 13/12/2011

For most of the projects, the signature of this declaration could be done directly via the IT reporting tool through an adapted IT mechanism.

1 If either of these boxes below is ticked, the report should reflect these and any remedial actions taken.

© Copyright 2011 EU-ADR Consortium 2

1 Publishable summary

www.euadr-project.org

During its third period, the EU-ADR project has continued its progress towards the production of the main results expected from the core work packages. The project did successfully pass its second review with the help of external independent experts during the period. It has reinforced its visibility and the contacts established with key stakeholders (including regulatory authorities and other projects), and has developed plans for long term sustainability. EU-ADR has actually become a seminal project, spinning out several other projects (mainly funded via FP7-Health) that apply EU-ADR technologies and methods to the study of specific drug and vaccine safety problems.

On the scientific and technical front, the most important advances in the project have been the nearly completion of work in core WPs, and a progressive shift towards a ‘system view’ of the development efforts, by which the different pieces (signal detection, substantiation, evidence combination, validation, exploitation) have been integrated and the focus has moved to the challenges that such integration presented. This process has been helped by a ‘cycle-based’ configuration of the workplan that, superimposed to the original workplan, has helped steer efforts through a common, ‘pipeline’ framework addressed to reproduce the sequential steps of the system, revealing input-output relationships and dependencies.

Notably, the project has also made great progress in devising a strategy for long term sustainability.

Particular outputs that deserve to be highlighted include: Reappraisal and rework of the set of true-positive and true negative signals, now named

‘reference sets’, so as to maximise their usefulness, especially for the signal detection method performance asessment and the validation activities.

Addition of three new events to the priority list of ten: Hip Fracture, Pancreatitis and Progressive Multifocal Leukoencephalopathy. Event definition for these events, and completion of terminology mapping, data extraction, and query harmonisation tasks related to all events.

Development and testing of algorithms and methods for signal detection. Tackling of studies on how parametrisation of methods affect their performance.

Development and completion of signal substantiation methods, implemented as web services.

Development of evidence combination methods based on Dempster-Shafer theory. Implementation of tools allowing visualisation, exploration and user interaction with such methods.

Development of the integrated EU-ADR Web Platform, which allows drug-event dataset management and sharing, visualisation, comparison and exploration of signal detection methods results, of signal substantiation results, and of evidence combination results, including custom user weightings given to the different methods based on belief and plausability.

Progress in system validation using both retrospective and prospective approaches, comparing results with those of spontaneous reporting systems. Tackling of event validation activities, addressed to verify the reliability of the events extracted from the databases.


http://www.euadr-project.org/

Overall, the project progress has been substantial, and the Consortium believes that it has allowed EU-ADR to become a breakthrough, reference project not only in the application of IT to pharmacoepidemiology, but also more generally in relation to demonstrating the feasibility of a federated model for re-using massive amounts of existing EHR data for research purposes; a model that, while respecting local and national ethico-legal limitations, allows to unleash the power of available medical information on millions of patients.


2 Core of the report for the period: Project objectives, work progress and achievements, project management

2.1 Project objectives for the period

The overall objective of the EU-ADR project is the design, development and validation of a computerized system that exploits data from electronic healthcare records and biomedical databases for the early detection of adverse drug reactions. The EU-ADR system intends to generate signals using data mining, epidemiological, computational and text mining techniques, and subsequently substantiate these signals in the light of current knowledge and understanding of biological mechanisms. The system should be able to detect signals better and faster than spontaneous reporting systems and should allow for identification of subpopulations at higher risk for ADRs. For the system to operate as an adjunct to safety reviewers during signal evaluation and follow-up, it should also enable easy access to the underlying data sources, allowing to quickly focus on information that is pertinent to a suspected ADR.

In this project, electronic healthcare records (EHRs) comprising demographics, drug use and clinical data of over 30 million patients from several European countries are available. These EHR databases form the foundation of the project, insofar as they supply the patient data on top of which the system is built.

A number of designs and techniques are used to process these electronic medical records. One of the objectives of this project is to study and compare a number of different techniques that, in essence, all aim to detect unexpected or disproportional rates of events.

Once generated, the signals are substantiated to place them in the context of the current biomedical knowledge. Essentially, this means searching for evidence that supports causal inference of the signal. The list of signals will be assessed by automatically investigating feasible paths that connect the drug and the adverse reaction involved in the signal. The general strategy is the automatic linkage of biomedical entities (drugs, proteins and their genetic variants, biological pathways, and clinical events) by means of data mining approaches and in silico predictions based on biomolecular structures.

The signal detection and substantiation algorithms are integrated in a computerized ADR detection and monitoring system. This involves the development of an evidence weighting scheme to combine the various pieces of information and present the user with a final list of ranked signals.

The system is expected to be tested retrospectively using test sets that are based on recent literature, including both known side effects and spurious signals. The system’s ability to rediscover drug-event combinations from the test set with known side effects provides an indication of the sensitivity of the system. The system’s ability not to signal drug-event combinations from the test set with spurious signals provides an indication of the specificity of the system.

After the system has been validated retrospectively, a prospective evaluation is also among the original objectives, centred on further investigating the top-ranking signals generated by the system.

The ultimate objective of the project is to demonstrate that an earlier detection of adverse side effects of drugs is possible using EHRs.


There is no definition of project objectives per period in the EU-ADR Description of Work. For our second report, submitted last year, we distilled from the work plan what the project was expected to deliver during its second period. Activities in this third project period were obviously expected to extend and deepen the tasks of the previous periods, and result in the bulk of outputs expected from the project.

For what refers to WP1, the third period of the project was expected to continue the maintenance of appropriate communication and work dynamics among partners and with external stakeholders. It was also expected to entail continuing ethical surveillance, and, following the experience of the previous period, repeat the project assessment exercise and compare the results obtained.

WP2 was officially finished in previous periods; however, the previous period already showed that a re-assessment of the true negative signals in the test set was needed. In this period, it was also planned to re-assess as needed the test sets, in case the results obtained from other WPs showed any important shortcoming in those sets. WP2 was also expected to cover as needed terminology mapping exercises derived from new events of interest beyond the 10 selected by the project, time and resource permitting.

The WP2/WP3 task force created in the previous period was also expected to continue its work in the third period, harmonising queries among databases by monitoring and optimising the actual codes used in each case to detect a specific event.

For WP3, and once the use of Jerboa as platform to extract, aggregate and export data was well established, work was expected to advance in signal detection decisively, so that a ‘final’ set of methodologies and parameters could be obtained in this third period.

Similarly, WP4 was expected in this period to complete the development of the different web services used to filter and substantiate signals output by WP3.

Additionally, this period was central for WP5, including the development of both evidence combination tools and the EU-ADR platform itself, including re-evaluation of design decisions as needed to enable full development of the system.

Although not expected to be completed in the period, activities in WP6 (System Validation) were expected to quickly unfold and show results as the full system ‘pipeline’ became progressively available.

For what refers to WP7, and aside from deepening the communication activities, with specific emphasis on liaison with relevant stakeholders and elaboration of relevant scientific publications, the focus was on developing the exploitation strategies, defining post-project use scenarios for the long term sustainability of the EU-ADR system.

Finally, management activities in WP8 were expected to consistently support all other activities in the project, ensuring efficiency and compliance with contractual requirements.


2.2 Work progress and achievements during the periodWP1: Scientific CoordinationWP leader: Johan van der Lei – EMC

Work for WP1 has continued as expected during the third project period, providing overall scientific leadership to the initiative, steering the project and balancing scientific excellence with pragmatic approaches. As in previous periods, cross-talking between WPs has been a specific concern, moreso as the different results in the central technical WPs (3, 4, 5) were progressively evolving towards their ‘almost-final’ shape – accordingly, system integration has been a continuing focus in the period.

A noteworthy activity in the period, developed in collaboration with WP8, was the devising of development ‘cycles’ to promote alignment of activities across WPs. This was necessary because, as WPs 3, 4 and 5 progressed, it was noted that there could be mismatches in the input-output relationships between them, derived from the different stages of development of each WP, a situation that could hardly be avoided taking into account the significant overlapping in the time schedules of these WPs as set out in the original plan. Therefore, three development cycles were devised (“silver”, “gold” and “bonus”), so that all WPs could follow the intended system ‘sequence’ from terminology mapping, to query harmonisation, signal detection, substantiation, evidence combination, system integration and validation. This helped clarify the dependencies among WPs, highlighted the iterative nature of the project developments, and served to test the system pipeline in order to detect bottlenecks or specific risks that could feed the exploitation plan. As an example, the cycles time schedule, which was updated during the period as needed, was originally devised as depicted below:

SilverCycle

Gold Cycle

Bonus

WorkplanWorkplan and statusand status

jul 10 aug 10 sep 10 oct 10 nov 10 dec 10 jan 11 feb 11 mar 11 apr 11 may 11 jun 11 jul 11

WP2 Terminology mapping

WP2/WP3 Query Harmonisation

WP3 Database extraction

WP3 Data Mining

WP4 Substantiation

WP5 Evidence Combination

WP5 System Development

WP6 Validation

Add columnsto xls

Add columns to xls

Integrate WP4 workflows Integrate WP5

Compare WP3 listwith FDA SRS

UGIB, AMI , ARF, AS, BE, RHABDDatabases use 'old' data (until 2008, in some cases until 2005 only)J erboa version 3.0 (incl. GPS + LEOPARD) - drug event-associations list (QResearch excluded) WP4 to prioritise associations with a threshold of post RR>=2Validation uses WP3 list directly (no filtering, no substantiation)

EUEU--ADR cycles planADR cycles plan

+ ALI , NEUTROP, PANCYTOP, CARDFIBDatabases use most recent data (ideally until end of 2009)Improved, 'Gold' J erboa

PML, Acute pancreatitis

Initial runs + harmonisation

Add columnsto xls

Add toxls

Gold J erboadeveloped Output from DBs

WP3 list complete

Integrate WP4 workflows

Integrate WP5

Compare list FDA + SRS +others

Terminology2 new events


D3.3 D4.4 D4.5 D6.2

D5.3 D5.4

‘Blank' spaces between cycles for any given WP is time that can be

used to refine methods, technologies, etc.

D6.3

D1.4D6.4D7.4D7.5

Add columnsto xls

Add columns to xls

Integrate WP4 workflows Integrate WP5

Compare WP3 listwith FDA SRS

UGIB, AMI , ARF, AS, BE, RHABDDatabases use 'old' data (until 2008, in some cases until 2005 only)J erboa version 3.0 (incl. GPS + LEOPARD) - drug event-associations list (QResearch excluded) WP4 to prioritise associations with a threshold of post RR>=2Validation uses WP3 list directly (no filtering, no substantiation)

EUEU--ADR cycles planADR cycles plan

+ ALI , NEUTROP, PANCYTOP, CARDFIBDatabases use most recent data (ideally until end of 2009)Improved, 'Gold' J erboa

PML, Acute pancreatitis


Add columnsto xls

Add toxls

Gold J erboadeveloped Output from DBs

WP3 list complete

Integrate WP4 workflows

Integrate WP5

Compare list FDA + SRS +others

Terminology2 new events


D3.3 D4.4 D4.5 D6.2

D5.3 D5.4

‘Blank' spaces between cycles for any given WP is time that can be

used to refine methods, technologies, etc.

D6.3

D1.4D6.4D7.4D7.5

The different cycles were defined according to the events that were included, the data coverage used in the process, and the Jerboa version used to extract and pool the data. Overall, the definition of


development cycles has proven invaluable to focus the efforts of partners and achieving the expected results.

It is to be remarked that the last cycle (“bonus”) was explicitly devised as an optional activity, as it resulted from interactions with relevant stakeholders. Indeed, the attendance of representatives of the Dutch national regulatory authorities to one EU-ADR meeting resulted in a ‘request’ to explore what the project could contribute to two events of recent interest regarding drug safety (Progressive Multifocal Leukoencephalopathy and Acute Pancreatitis). The project agreed to look into these events after the bulk of the work with the ‘committed’ ten main events was complete. As the period elapsed, and given some of the difficulties encountered with these events, it was agreed in the Consortium to also look at Hip Fracture, as additional event of interest.

Aside from strengthening the relationships with regulators, the project has also continued to ‘spin-out’ other projects in the FP7 Health and other programmes, in which subsets of partners participate. This is the case of VAESCO and SAFEGUARD, who together with the ongoing SOS and ARITMO projects, make use of EU-ADR technology to implement specific safety studies. At the end of the period, the project has also established solid bonds with the very relevant EHR4CR IMI project; this has resulted in a joint application to the recent IMI call. Relationships are also ongoing with the EMA-coordinated EnCePP initiative. Importantly, collaboration with US initiatives (FDA Sentinel, OMOP) has been reinforced during the reporting period (including mutual visits), and the lead developer of Jerboa won the international “OMOP Cup” competition thanks to one of the algorithms for signal detection developed in EU-ADR.

In the previous period, an interim assessment of the progress of the project (Deliverable D1.3) was elaborated. For this, two complementary methods were used. Firstly, an ‘objective’ assessment was carried out following standard project management practices for progress evaluation (earned value), considering both cost and schedule performance. Secondly, a survey was carried out among partners to provide a ‘subjective’ assessment counterpart, by scoring the degree of achievement of each original objective on a 0-10 scale. Both exercises yielded a progress estimation of around 64% of the project completed at the end of month 24.

The exercise was repeated at the end of the period currently being reported, in order to compare the results and evaluate the interim progress achieved. Using the same earning rule (50/50), the ‘objective’ estimation at the end of July 2011 yields a “progress” (Budgeted Cost of Work Performed, BCWP, or Earned Value) of 89% of the project’s budget. For comparison, taking into account that the project was granted during the period a 6-month extension (see Management section of this report for details), a linear time reference at month 42 over 48 months of total duration means that by July 2011 87.5% of the project had elapsed. However, the bulk of activities were expected to be completed by that date, and the estimated BCWS (Budgeted Cost of Work Scheduled) is actually 96.7%. Therefore, it could be concluded that the project was behind schedule (Schedule Performance Index, SPI, of 0.92, where unity means perfectly on track). However, it is noteworthy that both WP1 and WP8 (and, to a lesser extent, WP7) are continuing WPs throughout the project, and in these cases the 50/50 earning rule artificially reduces the earned value estimation at the late stages of the project. Taking into account only WPs 2 to 7, the BCWP is 94%, while the BCWS is 98% - for an improved SPI of 0.95. Looking at the costs incurred in to achieve this progress, the calculations (pending approval of the period costs by the EC) show an Actual Cost of Work Performed (ACWP) of 101% of the budget, which clearly indicates that the project is over budget (Cost Performance Index, CPI, of 0.88). Taking into account WPs 2 to 7 only, the CPI improves to 0.92, which indicates only a slight overspending compared to the original budget. The


‘objective’ analysis, therefore, shows the project slightly behind schedule and over budget, but reasonably on track.

When looking at the ‘subjective’ assessment, partners (n=18) score the degree of achievement of the objectives with a mean 8.5 (which can be interpreted as 85% of the project completed):

WP1 WP2 WP3 WP4 WP5 WP6 WP7 WP8 Global8.9 8.9 8.4 8.0 7.8 7.1 8.2 9.3 8.5

Indeed, 85% is not very different from the 89% BCWP mentioned above, what would confirm the alignment of both independent analyses, already observed in the first exercise done in the previous period. In this case, however, focusing only on WPs 2-7 (as the scores for WP1 and WP8 tend to indicate “quality” more than “progress”) reduces the global mean to 8.2, which is a bit of a departure from the 94% BCWP indicated by the equivalent ‘objective’ calculation. A possible explanation of this perception of less progress than what the objective analysis expresses can be derived from the very nature of science, according to which no endeavour really starts from scratch, and none really ever ends; extreme scores might thus be avoided because each objective is measured by some partners compared to an “ideal” result, as opposed to compared to what was committed in the Grant Agreement. Accordingly, it would be plausible that at the end of the project repeating the same exercise could yield a result well below 100%, even if all deliverables are completed and approved and the project is officially deemed as successfully finished. It can be concluded that this is thus a limitation of the methodology in academia-driven, scientific projects. Nevertheless, there is some significant variation in the scores among partners, with some apparent outliers; it is noteworthy that the global mode of the scores, considering only WPs 2-7, is 9, what is again essentially aligned with the ‘objective’ assessment results.

Overall, scientific direction in EU-ADR has accomplished all of its intended objectives in the third period. As in the previous periods, the planned dual leadership structure with the project management (WP8) has proved to be a key asset for the project, helping it to consistently advance in pursue of the intended goals, while allowing the flexibility needed to be able to react to bottlenecks and threats, and maximally exploit opportunities and synergies.


WP2: StandardsWP leader: Annie Fourrier-Réglat / Nicholas Moore - UB2

Work in WP2, aimed at providing standards to test and validate the system and helping to use databases together, was deemed to be mostly completed in previous periods, according to the original plan. This work entailed the definition of a ranked list of events of interest, the elaboration of ‘validation sets’ of drug-event pairs (both ‘true positive’ and ‘true negative’), and terminology mapping activities (across the four coding systems used in databases in the project, via UMLS) for the events of interest.

In previous periods, the event sets were identified (a total of 23 events were selected, of which the project decided to focus on the top 9 events: Bullous Eruptions, Acute Renal Failure, Anaphylactic Shock, Acute Myocardial Infarction, Rhabdomyolysis, Aplastic Anemia, Neutropenia, Cardiac Valve Fibrosis, Acute Liver Injury, plus Upper Gastrointestinal Bleeding as primary working example), the validation sets were elaborated for the top 6 events (the ‘true negative’ set was re-assessed after the first project review), and terminology mapping was finalised for 18 events.

During the last period, validation sets for the remaining 4 events (Acute Liver Injury, Cardiac Fibrosis, Neutropenia and Pancytopenia) were developed, and terminology mapping for the three ‘bonus’ events (Progressive Multifocal Leukoencephalopathy, Pancreatitis, Hip Fracture) was carried out.

However, the most important task later in the period was a full re-assessment of the ‘validation sets’ (now renamed as ‘reference sets’), for the 10 events of interest. The rationale for this late exercise was based on the problems that were being progressively encountered in other WPs with the original sets, especially as the validation sets were key to assess the performance of the system signal detection methods, its sensitivity and specificity, compared also to spontaneous reporting systems. The need for a late re-assessment has shown the Consortium the complexity of building ‘proper’ and stable reference sets to be used as gold standard.

The new reference sets were constructed with several partners and the process designed after many conference calls (see Figure 1 below). A stepwise approach was employed to identify which among a list of drug-event associations are previously well-known (‘true positive’ associations) or highly unlikely (‘true negative’ associations) based on published scientific literature, drug product labels, spontaneous reports made to pharmacovigilance database systems, and expert opinion. Only drugs with adequate exposure in EU-ADR to allow detection of an association were considered. Manual verification of true positive and true negative associations was independently performed by two researchers with proficiency in clinical medicine, pharmacoepidemiology, and pharmacovigilance. A panel of experts adjudicated equivocal cases and arbitrated any disagreements between evaluators.

The final reference sets comprise 94 drug-event combinations, which include 44 true positive associations and 50 true negative associations for the 10 events of interest (see Tables 1 and 2 below). For Cardiac Valve Fibrosis, there was no drug with adequate exposure in the database network that satisfied the criteria for a true positive association. Construction of the reference set was conducted by EMC in close collaboration with the WP2 partners (UB2) and the other WP3 partners, and reported (as WP2 was officially finished) as part of Deliverable D3.3. It was presented at the International Conference of Pharmacoepidemiology and the International Society for Pharmacovigilance in Istanbul, 2011 and is drafted as manuscript.


Figure 1. Flowchart showing the process of the construction of the Reference Set.

Table 1. True positives – Final list

Event Type Drug ATC Drug name Event

Type Drug ATC Drug name

ALI N03AF01 Carbamazepine CARDFIB

No drug with sufficient exposure that satisfies PubMed criterion for True Positive

ALI N03AG01 Valproic acid CARDFIBALI M01AX17 Nimesulide CARDFIBALI J01CR02 Amoxicillin and clavulanic acid CARDFIBALI A07EC01 Sulfasalazine CARDFIBAMI M01AH02 Rofecoxib NEUTROP H03BB02 ThiamazoleAMI A10BG02 Rosiglitazone NEUTROP B01AC05 Ticlopidine


Event Type Drug ATC Drug name Event


AMI G03AA07 Levonorgestrel and estrogen NEUTROP C09AA01 Captopril

AMI N02CC01 Sumatriptan NEUTROP N03AF01 Carbamazepine

AMI M01AH03 Valdecoxib NEUTROP N03AG01 Valproic acid

ARF C09AA01 Captopril PANCYTOP B01AC05 TiclopidineARF M01AE01 Ibuprofen PANCYTOP N03AF01 CarbamazepineARF N02BE01 Paracetamol PANCYTOP H03BB02 Thiamazole

ARF J01MA02 Ciprofloxacin PANCYTOP M04AA01 Allopurinol

ARF N05AN01 Lithium PANCYTOP C09AA01 Captopril

AS B01AC06 Acetylsalicylic acid RHABD C10AA07 RosuvastatinAS N02BE01 Paracetamol RHABD C10AA05 AtorvastatinAS J01CA04 Amoxicillin RHABD C10AA03 Pravastatin

AS J01MA02 Ciprofloxacin RHABD C10AA01 Simvastatin

AS M01AB05 Diclofenac

BE N03AF01 Carbamazepine UGIB N02BA01 Acetylsalicylic acid

BE J01EE01 Sulfamethoxazole and trimethoprim UGIB M01AB01 IndometacinBE N03AX09 Lamotrigine UGIB B01AB01 Heparin

BE M04AA01 Allopurinol UGIB H02AB06 Prednisolone

BE C03CA01 Furosemide UGIB M01AE01 Ibuprofen

Table 2. True negatives – Final list

EventType Drug ATC Drug name Event


ALI R03AC13 Formoterol CARDFIB N06AB08 Fluvoxamine

ALI S01ED05 Carteolol CARDFIB L04AX03 Methotrexate

ALI G04CA03 Terazosin CARDFIB C09CA04 Irbesartan

ALI N04BA02 Levodopa and decarboxylase inhibitor CARDFIB C03CA01 Furosemide

ALI C01DA02 Glyceryl trinitrate CARDFIB G03CA03 EstradiolAMI A10AD01 Insulin (human) NEUTROP C07AA07 Sotalol

AMI B03AA07 Ferrous sulfate NEUTROP H03AA01 Levothyroxine sodium

AMI J01CR02 Amoxicillin and clavulanic acid NEUTROP C10AA05 Atorvastatin

AMI J05AB11 Valaciclovir NEUTROP C10DA14 Isosorbide Mononitrate

AMI C10AB04 Gemfibrozil NEUTROP G04CA02 Tamsulosin

ARF R01AD09 Mometasone PANCYTOP C09CA04 IrbesartanARF H03AA01 Levothyroxine sodium PANCYTOP C10AA04 Fluvastatin

ARF R06AX26 Fexofenadine PANCYTOP S01EE01 Latanoprost

ARF N04BA02 Levodopa and decarboxylase inhibitor PANCYTOP S01ED01 Timolol

ARF B03AA07 Ferrous sulfate PANCYTOP R06AX27 Desloratadine

AS N06AX11 Mirtazapine RHABD G03CA03 Estradiol

AS H03AA01 Levothyroxine sodium RHABD C02CA04 Doxazosin

AS C02AC01 Clonidine RHABD A10BB12 Glimepiride

AS C02CA04 Doxazosin RHABD S01ED01 Timolol

AS N05BA04 Oxazepam RHABD C01DA02 Glyceryl trinitrate

BE C01BC03 Propafenone UGIB R06AX26 Fexofenadine

BE C07AB03 Atenolol UGIB C10AA01 Simvastatin

BE R03BB01 Ipratropium bromide UGIB S01EC03 Dorzolamide


EventType Drug ATC Drug name Event


BE R03BB04 Tiotropium bromide UGIB L02AE03 Goserelin

BE C08CA02 Felodipine UGIB N05CF01 Zopiclone

ALI: Acute Liver Injury; AMI: Acute Myocardial Infarction; ARF: Acute Renal Failure; AS: Anaphylactic Shock; BE: Bullous Eruptions; CARDFIB: Cardiac Valve Fibrosis; NEUTROP: Neutropenia; PANCYTOP: Aplastic Anemia/

Pancytopenia; RHABD: Rhabdomyolysis; UGIB: Upper Gastrointestinal Bleeding

WP2/WP3 Task Force: Harmonization of data extractionTo harmonize the process of data extraction concerning the events of interest in EU-ADR, the need for a WP2/WP3 task force was already detected in previous periods. The task force was constituted and chaired by Gianluca Trifiro’ (EMC), with active participation of Preciosa Coloma (EMC), Rosa Gini (ARS), Paul Avillach (UB2), and all the database owners. The aim of this task force was to ensure homogeneous data extraction and analysis among all the eight databases for the events of interest.

Work in the Task Force was prioritised according to the development ‘cycles’ envisaged in the period, and therefore focused on:

SILVER Cycle: AMI - Acute Myocardial Infarction, ARF - Acute Renal Failure, AS - Anaphylactic Shock, BE - Cutaneous Bullous Eruptions, RHABD – Rhabdomyolysis, UGIB - Upper Gastrointestinal Bleeding

GOLD Cycle: ALI - Acute Liver Injury, CARDFIB - Cardiac Valve Fibrosis, NEUTROP - Neutropenia – Agranulocytosis, PANCYTOP - Aplastic Anaemia – Pancytopenia

BONUS Cycle: HIP - Hip Fracture, PANCREAT – Pancreatitis, PML - Progressive Multifocal Leukoencephalopathy

As a first step, the database owners were asked to share the information on the queries executed to extract the data about the above-mentioned events for the three tables required by Jerboa (patient file, event file, drug file). In particular, the query for the event extraction was carefully analysed to assess the consistency among different databases. For each event, a structured table was built to collect in a logic and useful way all the information contained in the queries. Guidelines for query mapping were produced in order to make this process as objective as possible.

Two main sections have been considered in the queries:1. List of disease codes and keywords for search in free text in specific sections of the

database. Occasionally, other additional information has been reported as search criteria (i.e. laboratory value), based on the availability of specific information in each database.

2. Description of refinement algorithm or eventual manual validation to increase, respectively, sensitivity and specificity of event detection.

If major differences in the query analysis were identified (especially among databases with similar structure), database providers were asked to find an agreement via discussion about the best possible query.

At the end of the period, this harmonization process of event extraction has been successfully finalized for ALL the events studied in the Silver, Gold and Bonus Cycles.


WP3: Signal GenerationWP leader: Johan van der Lei/ Miriam Sturkenboom – EMC

After significant progress in method and extraction software development in previous periods, WP3 has been centred in the period on:

1) Comparison of signal detection methods against the final reference set2) Effect of the change in parameter settings across the signal detection algorithms3) Methods for prime suspect selection

1) Comparison of signal detection methods against the final reference setThe new reference sets (see WP2 above) were used to assess the sensitivity and specificity of the different signal detection methods. The performance (sensitivity and specificity) of the signal detection methods was compared and delivered in the final signal detection Deliverable D3.3. Table 3 below describes the methods that are currently being used. Methods were developed by several sites (UNMIB, LSHTM, EMC) and programmed in Jerboa for final inclusion in the EU-ADR system. Several meetings were organized with the partners to discuss the different methods.

Table 3. Overview of the signal detection methods currently used in EU-ADR.

SRS methods Cohort methods Case-based methods

Frequentist

Proportional Reporting Ratio (PPR) Incidence Rate Ratio (IRR) Matched case-control (CC)

Reporting Odds Ratio (ROR) Self-Controlled Case Series

(SCCS)

Bayesian

Gamma Poisson Shrinker (GPS) Longitudinal GPS (LGPS)

Bayesian Confidence Propagation Neural Network (BCPNN)

Bayesian Hierarchical Model (BHM)*

Elimination of protopathic bias

LEOPARD

* Note: even though the BHM was currently only applied to the IRR, it can be applied to other types of estimates as well.

The methods performance was done on the experience of in total, 146,830,906 patient years of 20,042,652 subjects over a total of 10-15 years (see figure 2).

Figure 3 shows the area under the ROC curve for the different methods that have been employed for signal detection in EU-ADR. This figure shows that all methods perform better than random baseline, that the LEOPARD filtering for protopathic bias always improves performance (but less so for methods that are already performing well), and that performance of methods does not differ that much. In general LGPS and case-control adjusting for drug count seem to slightly outperform the other methods, although this is certainly not statistically significant. In general the performance of methods is high, with the best performing method achieving an area under the ROC curve of 0.83, and a sensitivity and specificity of 0.80 and 0.70, respectively. This is not surprising, as the reference set was limited to drugs with a large amount of exposure in the different databases.


Figure 2. Distribution of patient data per database over time.

Figure 3. Area under the ROC curve for all methods, with and without LEOPARD filtering. Combination across databases was performed by pooling data. Error bars indicate 95% confidence interval.

2) Effect of the change in parameter settings across the signal detection algorithmsFor all methods, the following specifications were used to define exposures and outcomes:


Incident events. Only the first occurrence of an event was considered. Patient time after an event was completely ignored. The main reason for this is that, in EHR data, it is often difficult to distinguish between a recurrence of an event, or whether a reference is made to the event that occurred earlier.

Run-in period of 365 days. In order to determine that an event is incident, some patient time has to be available before the event occurred. Hence, during the first year of observation subjects were not considered for events or exposure counts, but events during this so-called run-in period were used to determine whether later events were truly incident events. This run-in period was not used for children younger than one year at the start of observation.

Exposure window definition. Exposure to a drug was defined as the duration of the prescription, excluding the first day of the prescription. If two prescriptions of the same drug overlapped in time, the exposure was assumed to start the day after the first day of the first prescription, and end on the last day of the last prescription.

Age stratification. Whenever appropriate, age was stratified in 5-year age ranges. Independence of drug risks. Currently, every drug-event pair is evaluated separately. Co-

medication is not taken into account.

LEOPARD was considered to be potentially complementary to all methods, and was therefore applied as a filter to the output of each method. LEOPARD can be applied at the level of the individual drug, but it can also be applied to a group of drugs. By grouping drugs with the same 4 higher level ATC digits (i.e. drugs with the same indication), LEOPARD has proven more able to detect protopathic bias. Signals that are flagged by LEOPARD either at individual or at group level were ranked lower in the list of signals than signals that were not flagged when calculating the AuC.

Currently we are testing the method performance along a wide range of parameter settings.

3) Methods for prime suspect selectionOne of the use cases of the EU-ADR system is the ranking of potential ADRs that require attention first. During the past months we have developed different strategies for the selection of prime suspects. The methods have been presented at the EU-ADR consortium meetings for discussion with the wider audience, and are still under development. A prime suspect has been defined as a drug-event combination that is not yet known, cannot be immediately explained by bias/confounding and affects many people. Methods for selection of prime suspects combine the number of excess cases, with enough exposure that pass the LEOPARD test and that do not seem to be confounded.


WP4: Signal Substantiation WP leader: Ferran Sanz / Laura Furlong – UPF

During the reporting period, the work focussed on finishing the development of software tools that accomplish the tasks required for the substantiation of the signal and subsequent evaluation of the such tools. The software tools include both methods implemented as web services and workflows that combine the web services.

The annotation of the EU-ADR corpus was finalized, and work was performed towards the use of the corpus for the development of a relation extraction system. Moreover, a lot of effort was put in the writing of manuscripts describing the work performed in the WP.

During the reporting period several teleconferences and web-seminars were organized to monitor the progress in each of the tasks. In addition, several face-to-face meetings were organized to address issues related to web services and workflow design and implementation. The progress in the individual activities is described hereunder.

Database and literature miningDuring this reporting period the annotation of the EU-ADR corpus was finalized. The details on the development of the EU-ADR corpus were provided in Deliverable 4.4: Report on literature and DB mining, and in a manuscript submitted for publication to the Journal of Biomedical Informatics.

The EU-ADR corpus consists of 300 Medline abstracts containing semantic annotations on biomedical entities and their relationships. The annotations were performed by domain experts who were capable of deciding if a text describes a relationship, and thus the corpus represents a “gold standard” dataset. To aid the work of annotators, a web-based annotation tool was developed to provide automatic annotation of the entities and propose relationships to be annotated.

The entities annotated were target (gene/protein, sequence variants), disorder (disease phenotypes of the adverse drug reactions), and drug (biologically active chemicals, marketed drugs and drug metabolites). The relationships considered were the following: target-disorder, target-drug and drug-disorder. Moreover, the level of certainty of each relationship was also specified by providing the relationship types:

Positive association (PA): the sentence clearly states that there is an association between the entities.

Negative association (NA): the sentence clearly states that there is no association between the entities.

Speculative association (SA): the sentence describes a putative relationship between the target and the disease. This might be confirmed or refuted later in the abstract, but in the sentence under study the relationship is presented as a speculation.

Once the expert annotation was finalized, work was conducted to harmonize annotations and evaluate the agreement between the annotators. The final EU-ADR corpus consists of the consensus annotations performed by the experts both at the level of entities and at the level of relationships. Based on all annotations of entities and relationships we analysed the number of entities and relations for whom a majority exists and in consequence were included in the final EU-ADR corpus (Table 4).


Table 4. Number of annotated entities and relationships and their agreement in the EU-ADR corpus. For the relationships agreement the second percentage shows the agreement given agreement on the entities.

Relationship type Entities Agreement Entities Relationships Agreement RelationsDrug-disorder 1849 1464 (79.2%) 655 294 (44.8%, 71.6%)Target-disorder 2356 1720 (73.0%) 822 380 (46.2%, 86.7%)Target-drug 2214 1701 (76.8%) 802 324 (40.4%, 68.5%)Overall 6419 4885 (76.1%) 2279 998 (43.8%, 75.6%)

In order to test the agreement of each annotator with the EU-ADR corpus we computed both the agreement statistics for both the entities (Table 5) and the relations (Table 6). The agreement figures show a good correspondence between the different annotations. From the results we can see that apart from annotator A4 all annotators show a good agreement with the EU-ADR corpus.

Table 5. Agreement between the annotators (A1-A5) and the automatic tool against the EU-ADR corpus for the annotated entities.

Relationship type A1 A2 A3 A4 A5 ComputerDrug-disorder 0.83 0.77 0.87 0.73Target-disorder 0.80 0.84 0.88 0.63Target-drug 0.82 0.83 0.87 0.67Overall 0.82 0.83 0.88 0.78 0.87 0.67

Table 6. Agreement between the annotators (A1-A5) and the automatic tool against the EU-ADR corpus for the annotated relationships.

Relationship type A1 A2 A3 A4 A5 ComputerDrug-disorder 0.75 0.51 0.83 0.69Target-disorder 0.85 0.72 0.71 0.57Target-drug 0.77 0.79 0.50 0.79Overall 0.80 0.75 0.62 0.51 0.83 0.66

In addition to comparing the annotations against the annotated corpus we also computed the inter-annotator agreement for each relationship (Table 7).

Table 7. Inter-annotator agreement statistics per relationship type.Drug-Disorder A1 A4 A5 ComputerA1 1.00 0.78 0.72 0.59A4 0.72 1.00 0.70 0.56A5 0.78 0.70 1.00 0.64Computer 0.59 0.64 0.64 1.00

Target-Disorder A1 A2 A3 ComputerA1 1.00 0.73 0.74 0.46A2 0.73 1.00 0.75 0.49A3 0.74 0.75 1.00 0.58Computer 0.46 0.49 0.58 1.00

Target-Drug A1 A2 A3 ComputerA1 1.00 0.78 0.75 0.49A2 0.78 1.00 0.74 0.52A3 0.75 0.74 1.00 0.58Computer 0.49 0.52 0.58 1.00

The agreement statistics are comparable with what has been shown in other annotation efforts2 3. The agreement on the entity annotation is a little higher than on the relationships. One reason for this is that it may be difficult for annotators to distinguish between a relationship being described in

2 Roberts A, Gaizauskas R, Hepple M, Davis N, Demetriou G, et al. (2007) The CLEF corpus: Semantic annotation of clinical text. AMIA Annu Symp Proc 2007: 625-629.3 Kolárik C, Klinger R, Friedrich CM, Hofmann-Apitius M, Fluck J. (2008) Chemical names: Terminological resources and corpora annotation. Proceedings of the LREC 2008 Workshop on Building and Evaluating Resources for Biomedical Text Mining : 51.


the text and the relationship actually being true. Even though a named entity recognition system has been used to suggest annotations to the annotators, we can see that the agreement between this system and the annotators is lower than the inter-annotator agreement. This means that the annotators modified the suggested annotations and were consistent on suggestions for change. Nevertheless, in our experience the use of a NER system is highly recommended to facilitate the annotation, since it made possible for the annotators to focus in the annotation of relationships.

The EU-ADR corpus can be downloaded from: http://euadr.erasmusmc.nl/sda/euadr_corpus.tgz. The annotation tool is available online at: http://euadr.erasmusmc.nl/sda/annotate.py.

Work in the period has also been directed to implement a system for the detection of relationships between biomedical entities from text. The system is based in the JSRE implementation from Giuliano and colleagues4, a Java implementation of a supervised machine learning approach developed for the identification of interactions between proteins, and achieves state-of-the-art performance. It is a kernel-based approach that uses only shallow linguistic information, such as tokenization, sentence splitting, Part of Speech tagging and lemmatization. The original JSRE system uses a linear combination of two kernel functions to represent the global context where entities appear and their local contexts. The global context kernel (GC kernel) considers the whole sentence to discover a relationship between two entities. The local context kernel (LC kernel) uses windows of predefined size around the entities to identify the roles of the entities within a relationship. The kernel-based machine algorithm used by JSRE is Support Vector Machines. We have added a third kernel to the JSRE system to incorporate deep syntactic information (D kernel) obtained from dependency parse trees to develop a method to detect the following relationships: target-disorder, target-drug and drug-disorder. The EU-ADR corpus was used for training and evaluation of the system. More details will be provided in a manuscript in preparation.

In a first step, we evaluated the contribution of using dependency parse information for the detection of the relationships of interest. The results for target-disorder relation are summarized in Table 8.

Table 8. Results for target-disorder relation.System Precision Recall F1 scoreOriginal JSRE (LC kernel + GC kernel) 0.72 0.61 0.66

Dependency kernel 0.76 0.60 0.65

Original JSRE + Dependency kernel 0.77 0.71 0.73

Original JSRE + Dependency kernel + keywords 0.77 0.70 0.73

These results indicate that the best system incorporates the 3 kernels without the use of association keywords. The association keywords are words or expressions that denote an association between the two entities, for instance the word bind is a keyword for associations between proteins. Similar results were obtained for the other two relationships (not shown).

Once the optimal configuration of the system was determined, it was used to train models for the detection of the above mentioned relationships. The purpose of these models is to distinguish the presence of an association between the entities from a mere co-occurrence without denoting an association. For that, we used the EU-ADR corpus and 10-fold cross-validation. The results are

4 Giuliano C, Lavelli A, Romano L. (2006) Exploiting shallow linguistic information for relation extraction from biomedical literature. : 5-7.


http://euadr.erasmusmc.nl/sda/annotate.py

http://euadr.erasmusmc.nl/sda/euadr_corpus.tgz

shown in the table below, and indicate good performance of the system for the detection of the relationships.

Precision Recall F1 score

Drug-Disorder 0.66 0.75 0.69

Drug-Target 0.78 0.70 0.73

Target-Disorder 0.72 0.67 0.69

Drug-target-pathway-adverse event mappingIn the reporting period, work was focused in finalizing and evaluating the developed tools (web services and workflows), and in writing manuscripts to report the results. Moreover, we finalized an analysis conducted on the DisGeNET database5 aimed at exploring the modular nature of human disease and also drug adverse reactions, which was published in PLoS One6. The details on the work towards the methodologies developed in this task were already reported in Deliverable 4.5: Report on Drug-Target-Pathway-Adverse event mapping, and is the subject of the publication entitled “Automatic filtering and substantiation of drug safety signals” currently under evaluation in the journal Plos Computational Biology. Here we provide an overview on the methodology for signal filtering and substantiation and present an example application. Details on the implementation of the web services and workflows can be found in Deliverable 4.5 and in the above-mentioned manuscript. The framework developed for the filtering and substantiation of drug safety signals consists of placing the signal in the context of current knowledge of biological mechanisms that might explain it. Essentially, we are searching for evidence that supports causal inference of the signal, i.e. feasible paths that connect the drug with the clinical event of the adverse reaction. The signal filtering analysis looks for evidence reporting the drug-event association in the biomedical literature and biomedical databases (Figure 4).

Figure 4. Schematic representation of the signal filtering process. Two workflows are available for the signal filtering process: the ADR-FM workflow uses a MeSH®-based approach to find drug-event pairs in Medline® citations, while the ADR-FD workflow uses text-mining to find the drug-event pairs in Medline ® abstracts, databases such as DrugBank and drug labels available at DailyMed®.

The signal substantiation process can be framed as a closed knowledge discovery process, analogous to the Swanson model based on hidden literature relationships7. We extend this framework by considering not only relationships found in the literature, but also those discovered by mining other data sources or found by applying different bioinformatics methods. For a drug-5 Bauer-Mehren A, Rautschka M, Sanz F, Furlong LI. (2010) DisGeNET: A cytoscape plugin to visualize, integrate, search and analyze gene-disease networks. Bioinformatics 26(22): 2924-2926.6 Bauer-Mehren A, Bundschus M, Rautschka M, Mayer MA, Sanz F, et al. (2011) Gene-disease network analysis reveals functional modules in

mendelian, complex and environmental diseases. PLoS One 6(6): e20284.7 Swanson DR. (1986) Fish oil, raynaud's syndrome, and undiscovered public knowledge. Perspect Biol Med 30(1): 7-18.


event association, we collect information about the targets of the drug by querying publicly available databases and by applying drug-target profiling methods8. In parallel, we retrieve information about the genes and proteins associated with the clinical event from a database covering knowledge about the genetic basis of diseases. Then, we combine these two pieces of information assuming that if the disease phenotype elicited by the drug is similar to the phenotype observed in a genetic disease, then the drug acts on the same molecular processes that are altered in the disease. Currently we consider two scenarios able to provide a causal inference of the signal (see Figure 5).

Figure 5. Schematic representation of the signal substantiation process. A. Signal substantiation through proteins. B. Signal substantiation through pathways.

First, we look for connections between the drug and the event through their associated protein profiles. Here, a connection is established if there are proteins in common between the drug-target and the event-protein profile (Figure 5A). Many ADRs are caused by altered drug metabolism for which genetic variants in metabolizing enzymes are often responsible. Consequently, we also consider drug metabolism phenomena as an underlying mechanism of the observed ADR by 8 Garcia-Serna R, Mestres J. (2010) Anticipating drug side effects by comparative pharmacology. Expert Opin Drug Metab Toxicol 6(10): 1253-1263.


assessing if the drug metabolites are targeting proteins that are known to be associated with the clinical event. The profile of targets of the drug and its metabolites is obtained by in silico profiling methods (Drug-Target-Profile). The profile of proteins associated with the clinical event is obtained by mining DisGeNET (Event-Protein Profile). The profiles are compared to find proteins in common in both profiles (Drug-Event Linking Proteins). The evidences that support the association of the drug and event with the Drug-Event Linking proteins are explored to determine if they support the causal inference of the signal.

Second, the association between the drug and the clinical event can involve proteins that are not directly associated with the drug and the clinical event, but indirectly in the context of biological networks. The final consequence of the drug action is the observed clinical event. Thus, proteins in the Drug-Target-Profile and in the Event-Protein Profile are searched in The Human Protein Atlas database to determine if they are expressed in the same tissue and cell type. Proteins that share expression at both levels (tissue and cell type) are used to query the Reactome database, and pathways that contain at least one protein from the Drug-Target-Profile and one protein from the Event-Protein Profile are retrieved. Then, these pathways are explored to determine if they support the causal inference of the signal (Figure 5B).

Figure 6. Integration of diverse biomedical sources and bioinformatics tools for the implementation of the filtering and substantiation frameworks. Data sources and bioinformatics methods relevant for signal filtering and

substantiation are accessed by means of SOAP web services and integrated using Taverna workflows.

Our approaches for signal filtering and signal substantiation were implemented using dedicated bioinformatic methods that are accessed through web services and integrated into processing pipelines by means of Taverna workflows (Figure 6). The substantiation workflow results can be visualized and analyzed by means of other bioinformatics tools such as Cytoscape9 (see Figure 7 and tables for an example results below), a software for network visualization and analysis. For the signal filtering process, we have implemented two Taverna workflows (ADR-FM and ADR-FD) that access data mined from databases such as DrugBank10, DailyMed11 and Medline® .(see tables below for example results). A third Taverna workflow, ADR-S, performs the signal substantiation process and was implemented by combining in silico target profiling, text mining and pathway analysis, among other bioinformatics approaches.

9 Shannon P, Markiel A, Ozier O, Baliga NS, Wang JT, et al. (2003) Cytoscape: A software environment for integrated models of biomolecular interaction networks. Genome Res 13(11): 2498-2504.

10 Wishart DS, Knox C, Guo AC, Cheng D, Shrivastava S, et al. (2008) DrugBank: A knowledgebase for drugs, drug actions and drug targets. Nucleic Acids Res 36: D901-906.

11 [Anonymous]. DailyMed. Available: http://dailymed.nlm.nih.gov/ via the Internet.


http://dailymed.nlm.nih.gov/

Figure 7. Cytoscape graph for QTPROL-haloperidol. The results of the ADR-S workflow can be visualized as a graph in which the nodes are proteins, compounds and clinical events. A: Detail of the network depicting the haloperidol targets, the proteins associated with QTPROL and the connection between them. The proteins encoded by the genes KCNH1, KCNH2 and CACNA1C constitute Drug-Event linking proteins between haloperidol and the terms corresponding to QTPROL. B: Detail of the targets of haloperidol, showing the adrenergic receptors (light blue) and the drug transporter encoded by the gene ABCB1 (magenta). In both graphs, the multiple edges between two nodes represent different evidences for the corresponding association between the nodes.

Table 9. Antipsychotics with low and high risk of producing prolongation of the QT interval (QTPROL) analyzed with the filtering workflows (ADR-FM and ADR-FD). For the ADR-FD, the individual results obtained from the three different sources used (Medline, DailyMed and DrugBank) are shown. The table shows the number of records found in each case. NA: Not Available.

WorkflowADR-FM ADR-FD

Risk of QTPROL

Drug Name ATC code MesH Medline DailyMed DrugBank

Low Sulpiride N05AL01 7 6 NA 0Quetiapine N05AH04 7 18 2 0Olanzapine N05AH03 14 20 1 0

High Ziprasidone N05AE04 15 38 3 0Pimozide N05AG02 0 16 0 0Haloperidol N05AD01 23 55 12 0

Table 10. Antipsychotics with low and high risk of producing prolongation of the QT interval (QTPROL) and the results of the substantiation process. The columns display the risk of producing QTPROL for each drug, the drug name, the ATC code of the drug, the proteins that explain the connection between the drug and the event (Drug-event linking proteins), the clinical events associated with these proteins (Events) and the drug activity values (pKi or pIC50) of each drug-event linking protein (average of the multiple values from different sources). For the drug-event linking proteins, the common protein name and the Gene Symbol (in parenthesis) are shown.

Risk of QTPROL

Drug Name ATC code Events Drug-event linking proteins

Drug activity values

Low Sulpiride N05AL01 None None None


Quetiapine N05AH04 LONG QT SYNDROME 1/2, 2, 2/5 and 2/3, TIMOTHY SYNDROME, Torsades de Pointes, Romano-Ward Syndrome

HERG (KCNH2) pKi 5.24

Olanzapine N05AH03 LONG QT SYNDROME 1/2, 2, 2/5 and 2/3, TIMOTHY SYNDROME, Torsades de Pointes, Romano-Ward Syndrome

HERG (KCNH2) pKi 4.64, pIC50 6.18

High Ziprasidone N05AE04 LONG QT SYNDROME 1/2, 2, 2/5 and 2/3, TIMOTHY SYNDROME, Torsades de Pointes, Romano-Ward Syndrome


Pimozide N05AG02 LONG QT SYNDROME 1/2, 2/3, 2 and 2/5, TIMOTHY SYNDROME, Torsades de Pointes, Romano-Ward Syndrome, cardiac arrhythmia


Cav1.2 (CACNA1C)

pKi 6.7

hEAG1 (KCNH1) pIC50 6.2Haloperidol N05AD01 LONG QT SYNDROME 2/3, 2, 2/5

and 1/2, TIMOTHY SYNDROME, Torsades de Pointes, Romano-Ward Syndrome


Cav1.2 (CACNA1C)

pKi 6.7

hEAG1 (KCNH1) pIC50 6.2

Availability of web services and workflowsADR-FM workflow: http://www.myexperiment.org/workflows/2280.html ADR-FD workflow: http://www.myexperiment.org/workflows/2279.htmlADR-S workflow: http://www.myexperiment.org/workflows/1988.html


http://www.myexperiment.org/workflows/1988.html



WP5: System IntegrationWP leader: José Luis Oliveira - UAVR

WP5 deals with the integration of outputs coming from other WPs. On the one hand, this means developing methods and systems to combine the evidence obtained from the signal detection methods of WP3 with the results from the different substantiation approaches in WP4; on the other hand, it entails the integration of heterogeneous computational systems and tools in a single working environment. This new integrated query environment, the EU-ADR Web Platform, is available at http://bioinformatics.ua.pt/euadr.

Evidence CombinationBoth WP3 and 4 accomplish their tasks by applying several different methods to judge if a signal is real, known, has known biology or in any other way is more or less likely to generate an ADR. In some cases all methods agree, and a true consensus can be found regarding the relationship at hand, but in most cases the different methods give different answers. Combination of evidence is the mathematical formulation that allows us to combine different sources of evidence under certain restricted assumptions.

Several methodologies for evidence combination (Voting, Bayesian, Dempster-Shafer) were carefully analysed (see Deliverable D5.3 for details). The ability to process both full and incomplete probability models is crucial to EU-ADR. Due to this, a decision was made to implement the Dempster-Shafer theories along with probability boxes for conversion of full probability models.

Implementation of Dempster-Shafer theory for evidence combination in EU-ADR was carried out during the period, including setting up a Dempster-Shafer object using conversion from Bayesian models, and setting up a Dempster-Shafer object for classifier models.

As a result, in WP4 each method supplies an evidence score, between 0 and 1, that represents the likelihood that there is a signal from data regarding the drug-event pair. This predicted evidence score is then used together with the results on the true positives and true negatives reported by WP2 to weight the predictive accuracy of each method.

The methods used by WP3 currently uses cutoffs on the risk ratios to form a classifier score (signal/ maybe signal/ no signal), which is weighted based on the validated data from WP2 in the same way as for WP4.

The output from the evidence combination package will be available in two ways on the EU-ADR web platform provided by WP5. There is a standard evidence combination that supplies belief and plausibility that there is a signal from WP3, WP4 and from a total combination of WP3 and WP4. The results give the user a way to judge the information at hand and the likelihood of an actual signal for a specific drug-event pair. Based on these results, the user can enter their own level of reliability for each method from WP3 and WP4 through a graphical user interface, allowing changes to be made that not necessarily reflect the views of the consortium. These changes allow the user to weight up or down methods that they believe in. The changed values are presented in another set of belief and plausibility columns in the same way as the standard combination. This last step is very important, because it adds information from the user which can be crucial to the analysis of the results for a specific drug-event pair.


http://bioinformatics.ua.pt/euadr

Technically, the methods for evidence combination in the EU-ADR project have been implemented in Java, originally to support EU-ADR through the workflow interface Taverna. The aim has been to deliver a flexible package that can be incorporated into various programs and platforms. The package is presented using the hierarchy of the code base and has been divided into core, GUI, xml-handling and interfaces. The full API is provided as an appendix to D5.3.

System IntegrationSeveral strategic decisions have been made along the project regarding technical infrastructure to be used in EU-ADR. All software components developed or reused inside the consortium were to be deployed as Web Services. Furthermore, they were wrapped in Taverna workflows for a straightforward usage. These choices were made following an exhaustive evaluation of several state-of-the-art solutions on software integration and interoperability, and are capable of fulfilling all the project’s requirements and streamlining its usage.

During the period, the EU-ADR Web Platform has been developed and implemented. The EU-ADR Web Platform is now a single workspace for the integrated analysis of drug-event pairs datasets. With this web-based tool, researchers can process entire datasets in tools provided by distinct partners in a single centralized environment. Additionally, they can perform systematic analysis tasks to assess the dataset ranking results and perform simple prospective evaluation operations. The entire platform is detailed in Deliverable D5.4, including the integration and interoperability requirements and solutions, its architecture and a detailed overview of the available features and user interface.


During the reporting period the WP5 work focussed mainly on the developments around three key topics:

1. Interoperability between the various heterogeneous components used in the EU-ADR architecture.For this interoperable software setup, a data-sharing standard was created, enabling communication between partners’ workflows and services. This communication, and consequent integration in the EU-ADR Web Platform, was only possible with the adoption of the common data exchange standard. Furthermore, the Taverna workbench provided a reliable workspace for workflow development and execution. This workflow engine can be used locally by EU-ADR researchers, and online through the EU-ADR Web Platform.

2. EU-ADR Web Platform.The final EU-ADR Web Platform is a distributed computerized system integrating a client, a server and a set of distributed Web Services. Users interact with the system through the application client, downloaded from the web platform server to end-users’ browsers. The client communicates with the platform server, which in turn invokes remote web services that feed the system with data and computational methods.

The core design goal of the web platform focused on integration of heterogeneous services developed by different partners – offering users a single transparent system which aggregates all project outcomes. Figure 8 presents an overview of the distributed system architecture, where users query a multitude of different databases through the web platform interface, without ever contacting third-party services directly.

Figure 8. EU-ADR system distributed architecture

To attain an interactive, fluent and smooth user experience, the Web Platform client was developed using the Google Web Toolkit alongside various technologies and software components that intend to facilitate GWT application development and provide a solid ground framework on which the client module could be built. Server-side, the system is service-oriented and takes advantage of the Java Servlet API to communicate with the client through opaque JSON remote procedure calls. Persistence to a MySQL database is assisted by Hibernate, the chosen ORM tool.


Taverna is the service composition and orchestration tool used to both construct and enact scientific workflows. Project services were wired together using the graphical workflow builder of the Taverna Workbench, and the Web Platform delegates the workflow execution on the Taverna CLI Tool, which invokes the required services and returns results in one or more XML files conforming to the EU-ADR project schema.

All the technological work behind this platform is reported in Deliverable D5.4.

3. Evidence combination integration into the platform. Evidence combination through the Dempster-Shafer method has been implemented as a service (web service and applet) and was integrated inside the platform to avoid external services invocation. The high level process employed to combine evidence is depicted in Figure 9, with focus on the inputs and output of the service.

Figure 9. Evidence combination process

The system features intend to offer users an appealing open workspace for adverse drug reactions study, which facilitates collaboration and promotes further research outside the platform – through links provided to external drug and publication databases. All system data and substantiation knowledge can be easily exported to open formats, which can later be imported to other analysis tools – making the web platform a valuable information resource usable in a larger investigation pipeline.

The following set of high-level system features have been identified and implemented in the EU-ADR Web Platform:

User account management: Invite-based user registration Login/Logout Recover lost password Change password

Dataset input/output: Import dataset from tabular file Create sample drug dataset from wizard Export dataset with substantiation data


Delete datasetDataset collaboration:

Share dataset with other users Revoke access to shared dataset/Abandon shared dataset EU-ADR project collaborative datasets workspace

Drug-event pair analysis: Substantiate one or more drug-event associations Substantiate whole dataset Substantiation evidence exploration Rank and filter drug-event pairs

Evidence combination: Combine substantiation evidence using Dempster-Shafer theory Customize parameters of Dempster-Shafer method per dataset Combined evidence exploration

Workflow execution scenarios: Remote workflow execution (on platform server) Local workflow execution (on user machine)

Support and documentation: Wiki-based documentation In-context help User feedback submission form

System monitoring: Fine grained user actions tracking Error logging and reporting to administrators

Example screenshots of the EU-ADR Web Platform user interface are shown below. The system will be refined as needed in the final project period according to user feedback.


WP6: System validationWP leader: Gianluca Trifiró - EMC

WP6 is addressed to iteratively perform retrospective and prospective validation of the EU-ADR system. In particular, the tasks of WP6 have been directed to compare the EU-ADR system with traditional spontaneous reporting systems with respect to signal detection. The comparison has been performed via retrospective validation, using the reference sets of true positive and true negative associations, and prospective validation, considering a restricted number of drugs that may be investigated in both systems. In addition, in the context of WP6, intense efforts have been put in the verification of the process of data extraction with respect to the events of interest in EU-ADR. This ‘event validation’ activity has been carried out via electronic medical records and hospital charts review.

Retrospective validationRetrospective validation is intended to compare signal detection performance between the EU-ADR system and traditional spontaneous reporting systems (SRS) databases, using the reference sets of true positive and true negative drug-event associations. An intermediate validation was performed using the original reference sets, which revealed some limitations. For this reason, new reference sets were created (see WP2 above) using new criteria (e.g. a specific amount of exposure in the EU-ADR database network is required for drugs involved in the sets). The final version of the reference sets has been discussed and approved by members from EMC, UB2, LSHTM, and AZ. Final retrospective validation has been completed and described in Deliverable D6.3. In general, comparable specificity and slightly lower sensitivity for signal detection in EU-ADR vs. SRS have been demonstrated. The Food and Drug Administration AERS and World Health Organization spontaneous reporting databases have been considered as comparators. Gold standard methodology for signal detection has been used in SRS databases. To make the analyses between EU-ADR and SRS as comparable as possible, UB2 mapped MEDdra terms (used in SRS for coding of events) to the UMLS concepts that have been already selected for data extraction from the databases involved in EU-ADR. Additional subgroup analyses will split the follow-up years based on the date of the first launch of the corresponding safety warning and other implementations of health policy interventions, to explore if any related change in the drug prescribing pattern may explain the lower sensitivity in EU-ADR as compared to SRS, when considering drug-event associations that have been known for a long time.

Prospective validationWith respect to prospective validation, we have compared the signal detection between EU-ADR vs. SRS databases considering all the drugs potentially associated with the top ten events of interest in the project. Preliminary analyses and results of intermediate validation have been described in Deliverable D6.2. EMC led this task; however, all the database owners have been contributing data and expertise for the interpretation of the results. In fact, pooled and database-specific list of signals from EU-ADR have been created based on different data mining techniques. Again, the Food and Drug Administration AERS and World Health Organization spontaneous reporting databases have been considered as comparators. In the next months also timing of signal detection will be explored and compared between EU-ADR and the SRS.

In the preliminary analyses, we restricted the comparison to a list of 453 drugs that could be mapped to ATC and were captured in both EU-ADR and SRS databases. In the next months, a number of sensitivity analyses will be carried out. In particular, signal threshold in EU-ADR will be changed and methods to further limit the potential effect of confounding will be tested.


Event validationAs an additional activity in this WP, intense efforts have been devoted to validation of events. This task is intended to verify both the quality of terminology mapping and the data extraction of the events of interest from the participating databases. Additionally, validation of the events may facilitate the conduction of traditional hypothesis testing studies for the validation of the top-ranked signals identified in EU-ADR, using common practice in pharmacoepidemiology. Case validation is in fact one of the most troublesome actions when performing an observational study using electronic medical record databases.

For this activity, questionnaires and related validation algorithms have been created for upper gastrointestinal bleeding, acute renal failure, acute myocardial infarction and bullous eruptions. For each event, the Chameleon tool for electronic data collection has been created, taking into account the event-specific questionnaire.

Random selection of 200 potential cases has been done in databases participating in this task and validation has been performed in each database by medically trained assessors, who were adequately instructed. At the time of this report, validation of UGIB is almost completed, and the task is ongoing for the other events. Once the validation will be completed, a different event definition may be considered and databases will be asked to identify new lists of event-specific signals, using the same statistical analyses but the new event definition. This sensitivity analyses will give us the opportunity to explore the possible effect of outcome misclassification.


WP7: Dissemination and Exploitation WP leader: Eva Molero - FIMIM

The objective of WP7 is to ensure optimal communication within the project and to disseminate information and knowledge generated by the project to relevant stakeholders. During the third period of the EU-ADR Project, WP7 has achieved the objectives planned. Most of the efforts have been devoted to the preparation and undertaking of dissemination activities (especially scientific papers), and to the design and discussion of different sustainability scenarios for long term maintenance of the EU-ADR system after the EC funding period within the task 7.3 Exploitation.

The progress in the specific WP activities is described below.

Dissemination ActivitiesThe updated Communication Plan (reported under Deliverable D7.3: Intermediate Report on Dissemination Activities) has been implemented satisfactorily, thus guiding all dissemination efforts in the project and ensuring its consistency and effectiveness.

The number of activities carried out by partners has kept a steady pace, reaching more than 230 activities since the beginning of the project, almost 90 of them corresponding to the last reporting period. 23 scientific articles have been published in indexed journals during the project. It is worth mentioning that 13 of these articles have been published in the last reporting period, which is consistent with the fact that the EU-ADR research has achieved a sufficient maturity to allow for a more intense publication activity. Participation in congresses either via posters or oral communications has also been remarkable.

A list of the manuscripts published during the period follows (the full list is available in the website):

1. Pirmohamed M., Friedmann P.S., Molokhia M., Loke Y.K., Smith C., Phillips E., La Grenade L., Carleton B., Amati-Papaluca M., Demoly P., and Shear CPT N.H. Phenotype Standardization for Immune-Mediated Drug-Induced Skin Injury. Clinical Pharmacology & Therapeutics 89, 896-901. DOI:10.1038/clpt.2011.79

2. Aithal G.P., Watkins P.B., Andrade R. J., Larrey D., Molokhia M., Takikawa H., Hunt C.M., Wilke R. A., Avigan M., Kaplowitz N., Bjornsson E. and Daly A. K. Case Definition and Phenotype Standardization in Drug-Induced Liver Injury. Clinical Pharmacology & Therapeutics 89, 806-815. DOI:10.1038/clpt.2011.58

3. Bauer-Mehren A, Bundschus M, Rautschka M, Mayer MA, Sanz F, et al. Gene-Disease Network Analysis Reveals Functional Modules in Mendelian, Complex and Environmental Diseases . PLoS ONE 6(6): e20284. DOI:10.1371/journal.pone.0020284

4. Coloma P., Schuemie M.J., Trifirò G., Gini R., Herings R., Hippisley-Cox J., Mazzaglia G., Giaquinto C., Corrao G., Pedersen L., van der Lei J., Sturkenboom M., on behalf of the EU-ADR Consortium. Combining electronic healthcare databases in Europe to allow for large-scale drug safety monitoring: the EU-ADR Project . Pharmacoepidemiology and Drug Safety, 20: 1–11. DOI: 10.1002/pds.2053

5. Schuemie M.J. Methods for drug safety signal detection in longitudinal observational databases: LGPS and LEOPARD. Pharmacoepidemiology and Drug Safety, 20: 292–299. DOI: 10.1002/pds.2051


http://onlinelibrary.wiley.com/doi/10.1002/pds.2051/pdf





http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0020284



http://www.nature.com/clpt/journal/v89/n6/full/clpt201158a.html




6. Garcia-Serna R, Mestres J. Anticipating drug side effects by comparative pharmacology. Expert Opin Drug Metab Toxicol (2010) 6(10): 1253-1263. DOI: 10.1517/17425255.2010.509343

7. Bauer-Mehren A, Rautschka M, Sanz F, Furlong LI. DisGeNET: a Cytoscape plugin to visualize, integrate, search and analyze gene-disease networks. Bioinformatics (2010) 26(22): 2924-2926. PubMed PMID: 20861032. DOI: 10.1093/bioinformatics/btq538

8. Arrais JP, Fernandes J, Pereira J, Oliveira J.L. GeneBrowser 2: an application to explore and identify common biological traits in a set of genes. BMC Bioinformatics. 2010, 11:389. DOI:10.1186/1471-2105-11-389

9. Vidal D, Mestres J. In silico receptorome screening of antipsychotic drugs. Molecular Informatics, 29: 543–551. DOI:10.1002/minf.201000055

10. Curcin V, Bottle A, Molokhia M, Millett C, Majeed A. Towards a scientific workflow methodology for primary care database studies. Stat Methods Med Res. August 2010 19:378-393. PubMed PMID: 20442191. DOI: 10.1177/0962280209359880

11. Wu TY, Jen MH, Bottle A, Molokhia M, Aylin P, Bell D, Majeed A. Ten-year trends in hospital admissions for adverse drug reactions in England 1999-2009. J R Soc Med. 2010 Jun; 103(6):239-250. PubMed PMID: 20513902. DOI:10.1258/jrsm.2010.100113

12. Molokhia M, Curcin V, Majeed A. Improving pharmacovigilance. Use of routinely collected data. BMJ. 2010; 340:bmj.c2403. PubMed PMID: 20442236. DOI: 10.1136/bmj.c2403

13. Matos S, Arrais JP, Maia-Rodriguez J, Oliveira JL. Concept-based query expansion for retrieving gene related publications from MEDLINE. BMC Bioinformatics. 2010; 11:212. DOI:10.1186/1471-2105-11-212

Regarding participation in congresses, the EU-ADR project has been presented among others in:

- eHealth Week 2010. March 2010. Barcelona, Spain.

- 46th Annual Meeting of the Drug Information Association (DIA). June 2010. Washington DC, USA.

- European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP) Plenary session June 10, European Medicines Agency, June 2010. London, UK.

- 16th World Congress of Basic and Clinical Pharmacology (WorldPHARMA2010). July 2010. Copenhagen, Denmark.

- 26th International Conference on Pharmacoepidemiology & Therapeutic Risk Management (ICPE). August 2010. Brighton, UK.

- 13th International Congress on Medical Informatics (Medinfo 2010). September 2010. Cape Town, South Africa.

- 32nd Spanish Congress of Pharmacology. September 2010. Leon, Spain.

- 9th European Conference on Computational Biology (ECCB). September 2010. Ghent, Belgium.

- 7th Annual Conference of Pharmaco-Epidemiology. September 2010, Avignon, France.

- 11th International Conference on Systems Biology (ICSB). October 2010. Edinburgh, Scotland, UK.


http://www.ncbi.nlm.nih.gov/pubmed/20426836


http://www.bmj.com/content/340/bmj.c2403.long

http://www.bmj.com/content/340/bmj.c2403.long



http://smm.sagepub.com/content/19/4/378.long

http://smm.sagepub.com/content/19/4/378.long

http://onlinelibrary.wiley.com/doi/10.1002/minf.201000055/full

http://www.biomedcentral.com/1471-2105/11/389

http://www.biomedcentral.com/1471-2105/11/389




- III International Symposium on Biomedical Informatics in Europe. October 2010. San Sebastian, Spain.

- 10th IEEE International Conference on Information Technology and Applications in Biomedicine (ITAB 2010). November 2010. Corfu, Greece.

- IV Annual Conference on Biomedical Research Technology Platforms: Innovative Medicines, Nanomedicine and Health Technology. February 2011. Madrid, Spain.

- Society of Toxicology (SOT) Annual Meeting 2011. March 2011. Washington, USA.

- 27th International Conference on Pharmacoepidemiology & Therapeutic Risk Management (ICPE) August 2011. Chicago, USA.

- 11th International Society of Pharmacovigilange 2011 (ISOP). October 2011. Istanbul (planned).

The objective of reaching extended target audiences has been achieved through several actions, such as the participation in the eHealth Week 2010 (Barcelona, Spain), the workshop organised in Medinfo 2010 (Cape Town, South Africa) and the continued interviews with general practitioners and patients in the UK.

Clustering activities with related initiatives has continued. It is noteworthy that the EU-ADR representative (M. Schuemie) won the OMOP Cup. The OMOP Cup is a competition to detect adverse drug reactions in a large set of simulated patient data, sponsored by the OMOP initiative (Observational Medical Outcomes Partnership) in the USA.

The partner’s dissemination activities have been regularly updated in a dedicated database. The specific procedures set for the internal review of publications (including manuscripts submitted to peer-reviewed journals and conference papers) are being implemented and plans for publications are addressed at each consortium meeting.

ExploitationThis activity focuses on studying different sustainability scenarios for long term maintenance and development of the EU-ADR system. During the two first years of the project, efforts centred on the creation and maintenance of a Results Inventory (i.e. ‘low hanging fruits’), published in the private zone of the project website. During the reporting period, efforts have continued towards gathering information on mature project results in order to update this Inventory.

However, efforts have gone well beyond that. From mid-2010, this activity significantly shifted gears towards studying the prospective use of the EU-ADR system -globally considered- after the end of the project. The first step was to conduct a brainstorming session in the framework of the 9th

EU-ADR Consortium Meeting (Florence, October 2010). As a result, several initial, potential exploitation scenarios were devised. These were presented and benchmarked at the 10th Consortium Meeting (Aveiro, February 2011). As a third step, and as discussions among partners were intensifying, a survey was conducted within the Consortium, which enabled to perform a SWOT analysis of the benchmarked scenarios. The results were presented at the 11 th Consortium Meeting (April 2011, Barcelona). As a result of this whole process, a first “EU-ADR Alliance” concept and proposal was devised by the project leadership on occasion of the eHealth Week 2011 (May 2011, Budapest), and subsequently submitted for the consideration of the Consortium in June 2011.


The EU-ADR Alliance represents a proposal for continuing the collaboration among EU-ADR partners once the contractual period of the project comes to an end. The starting point is the definite interest expressed by most of the EU-ADR partners in enabling a post-project phase in which the work started in EU-ADR could be continued. However, during the exploitation scenarios configuration process it became apparent that two limitations were paramount: a) most partners would be unable to risk non-compliance with their own local rules and regulations, or to jeopardise their own individual business models, and therefore a full commercially-oriented endeavour was not feasible; and b) that creating a formal supra-organisational structure would probably represent too much of an overhead and added complexity, de facto blocking the continuation of post-project activities, especially in the initial stages where uncertainty prevails.

In this context, the EU-ADR Alliance is conceived as a ‘lightweight’ collaboration environment composed by Members (initially, the willing EU-ADR partners) that are affiliated to the Alliance, and agree on specific rules and methods of working in its framework. The Alliance will have the goal of running studies and answering drug safety questions from Members and external organisations in a collaborative and federated manner, generally in those cases where the participation of more than one database would be required for the study (as otherwise, Members would naturally develop such studies individually, and retain in any case freedom to do so). The EU-ADR Alliance will also be devoted to maintenance and continuing improvement of the EU-ADR system. This scenario lies on some fundamental ideas, such the concept of “federated databases”, non-competition with the individual EU-ADR Alliance Members own research work and business opportunities, and independence and scientific interest as the driving force. A governance model has been devised, as well as the definition of activities the EU-ADR Alliance would perform, and the membership regulation.

It has also been proposed that a legal entity representing the EU-ADR Alliance for contractual and operating matters (the Alliance Coordination Office, ACO) would act as “coordination point” of the Alliance, guarantor of its working mechanisms, and as neutral mediator between the external institutions commissioning safety studies and the Alliance members for the performance of such studies. The figure of an EU-ADR Alliance Scientific Coordinator (ASC) has also been envisaged. The role of the ASC would comprise general Alliance leadership and coordination at the scientific and technical level, and promotion of contacts and relationships with other relevant initiatives and stakeholders.

It is expected that the EU-ADR Alliance would initially perform 5 kinds of activities: i) A yearly agreed Alliance Research Plan, devoted to performing internal research of common interest for the Members; ii) EU-ADR system maintenance and improvement, including the Web Platform, web services, the Jerboa data extraction software, etc.; iii) Commissioned studies; iv) Partners individual studies, and v) Coordination and management of the Alliance.

The EU-ADR partners interested in continuing the collaboration would become members of the EU-ADR Alliance through the signature of a Memorandum of Understanding (MoU). The Members of the EU-ADR Alliance would commit to maintain and improve the system, meaning at least regularly extracting and updating the data in the platform, keeping the EU-ADR platform in operation and maintaining the web services and other necessary software. It should be stressed that the Members will be free to decide in which studies they want/can participate on a case-by-case basis, and that the scientific direction of each specific study will be attributed to the most appropriate Member according to the required expertise.


Membership to the Alliance would also be open to new organisations that are not partners in the EU-ADR project. It is in fact envisaged that more EHR databases (and partners with complementary expertises) would be interested in joining the EU-ADR Alliance, what could allow the exploitation of healthcare data of potentially more than 100 million European citizens. So far, most EU-ADR partners have committed to participate in the Alliance (except UNOTT, who cannot participate due to governance restrictions). Encouraging contacts have started with the pharmaceutical industry and regulators on specific routes for collaboration, and the Alliance has planned to tackle a proof of concept phase by September 2011, piloting the operating mechanisms in the context of real-world opportunities, in order to fine tune the details of the implementation plan.

WP8: Project ManagementSee details in section 2.3 - Project Management below.


2.3 Project management during the periodProject Management in EU-ADR is essentially centred on WP8. During the third Reporting Period of the Project, management work has continued supporting all project activities with a view on adequate progress, comprising:

Follow-up of activities and monitoring of compliance with the work plan, planned resources and time schedule, promoting as far as possible the synergy between different activities and efficiency throughout, in close co-operation with the Scientific Co-ordination.

Support to the Co-ordinator in the liaison with the EC Project Officer. Mediation of quality control procedures (‘review’) on deliverables. Submission of project

deliverables to the EC and follow-up of milestones achievement. Consensus building activities, and mediation, especially for what refers to financial and

legal issues. Fostering communication within the Consortium. Continuous maintenance of organisation

charts, contact lists and a repository of important documents and tools for partners at the private zone of the EU-ADR website.

Meetings organisation: 6 General Consortium meetings from February 2010 to July 2011. Meetings minutes production, logistics arrangements, upload of presentations to the private zone of website.

Organisation and preparation of the second Review meeting in April 2010, which comprised a Consortium-only meeting rehearsal on the previous day. Analysis of the review meeting output.

Completion of administrative and financial periodic report for the second year of the project, including several iterations in response to the EC comments, entering data into the online NEF application, resolving issues, etc.

Financial management: payments calculation and distribution, follow-up of budget consumption, assistance to partners. Preparatory work for administrative and financial periodic reports of the third Reporting Period.

Risk management: Support to identification and assessment, devising of mitigation and contingency plans.

In close co-operation with the Scientific Co-ordination, concertation activities and promotion of linkage with other national and international initiatives.

For all of these tasks, the planned ‘tandem’ leadership structure with the Scientific Co-ordination has been key to ensure unified project steering and coherence throughout. Operatively, this is achieved by frequent communication between the Co-ordinator and the Project Manager (briefings by phone and email), supplemented by direct communication between members of the respective teams at EMC and FIMIM.

Of special note during the period, a Grant Agreement amendment was processed as a consequence of a request for a 6-month extension of the project. This extension was discussed within the Consortium and with the EC officer, and deemed to be desirable to optimise the project results (particularly, the methods for signal detection, the EU-ADR Web Platform and the validation activities) and improve the exploitation studies. The amendment involved gathering and preparing the documentation, managing the work plan changes derived from the extended time schedule, and communicating with partners to update some contact details. All the changes are included in an updated DOW, dated 25 May 2011.

The list of meetings held during the period is the following: 7th Consortium Meeting, Sitges (Barcelona), Spain. 15-16 February, 2010.


2nd Project Review Meeting, Brussels, 13th April, 2010 (preparatory Consortium meeting), 14th April, 2010 (Review meeting).

8th Consortium Meeting, Rotterdam, the Netherlands. 25-26 May, 2010. 9th Consortium Meeting, Florence, Italy. 25-26 October, 2010. 10th Consortium Meeting, Aveiro, Portugal. 2-3 February, 2011. 11th Consortium Meeting, Barcelona, Spain. 4-5 April, 2011. 12th Consortium Meeting, Barcelona, Spain. 22-23 June, 2011. (Scheduled) 13th Consortium Meeting, Barcelona, Spain. 5-6 September, 2011.

For all of these meetings, hosting has been supported, minutes recorded, and materials have been compiled and placed in the EU-ADR website. As in previous periods, frequent Consortium meetings have been instrumental in providing cohesion to the project and steering the efforts of the partners.

Additionally, workshops and WP-specific meetings have been held as needed (including teleconferences as appropriate), e.g. WP3 (14 Feb 2010, 4 Oct 2010), WP4 (5 April 2010), Event Validation (4 April 2010). Sometimes these have been organised as satellite meetings of general Consortium meetings for convenience. Furthermore, the forums section of the website has encouraged the interaction in the framework of specific WPs as well.

As in previous periods, WP Leaders have been given autonomy to organise the work within their respective WPs in the way they think is best. Coordination of WPs has been achieved via Consortium Meetings, and by direct interaction with the Scientific Co-ordinator and Project Manager. Specifically in this period, and as explained in the WP1 section above, a ‘development cycles’ scheme was devised and used to align activities of the different WPs around a common ‘system pipeline’.

The second review meeting was held in Brussels on 14 April 2010. The project was thoroughly evaluated by external experts, and recommendations were made. The meeting was very successful for the project.

As implied by section 3 above, remarkable progress has been made by the project during its decisive third period, in which the central WPs have delivered the bulk of its expected outcomes. No fundamental deviations affecting the objectives have been detected – although delays have continued to affect deliverable submission. This has been essentially due to a systematic solving of trade-offs in favour of quality of results, and the will to include the most recent results in each deliverable. A significant part of the delays have also been derived from the need to re-assess the validation sets defined early on in the project. This has been a worthwhile exercise and a valuable learning experience, and the Consortium is confident now that the final reference sets obtained are a definite improvement over the original ones.

Complementing the standard progress evaluation procedures in place in EU projects (deliverable submission, review meetings with independent experts), EU-ADR also initiated in the previous period its own interim assessment exercise, which showed a progress of 64% at month 24. The exercise has been repeated at the end of the current reporting period, showing that the project is slightly behind schedule and over budget, with an overall progress achieved of around 90% at month 42. Details can be found in the WP1 section above.

Current top risks affecting the project are attached as annex to this report.


The second interim payment was received from the Commission after final approval of the second period report and the corresponding cost justifications, and payments were distributed accordingly in October 2010.

Regarding dissemination and use of results, including the project website, the activity of the Consortium has been remarkable. Numerous dissemination activities have been undertaken and can be consulted in the private zone of the website. More information can be found in the WP7 heading in section 2.2 of this report.

The number of person-months spent in the project during the Feb 2010 – Jul 2011 follows generally the original plan and the trends observed in previous reporting periods. An effort breakdown per WP per partner is provided in Table 1 below. The total effort spent in RP3 is slightly over 240 person-months, 30% of the total effort planned for the project. Effort consumption has somewhat increased with respect to that of RP2, again reflecting the ‘performing’ phase that the Consortium has entered in a period in which the central deliverables of the core WPs were to be produced. Effort figures per partner reflect some heterogeneity derived from the different involvement of partners in the activities. Cumulative effort spending rises to 82% of the total effort planned.

As regards to budget consumption, the total costs incurred in during the third period (see form Cs attached) as reported by partners are currently estimated at over 2,378,999€ representing approximately 40,5% of the total project budget. Again, this represents a significant increase over the second reporting period (similar in percentage to that of the effort figures), but continues the trend observed in previous periods, by which budget consumption rates are higher than effort consumption rates in general. This indicates that the Consortium has made in this third period again a relatively more intensive use of permanent staff, more expensive than that initially budgeted for. Section 4 below shows that the vast majority of costs during the period correspond to personnel costs and travel and subsistence costs related with the project meetings.


TABLE 1. PROJECT PERSON/MONTHS TABLE

EMC

FIM

IM

UPF

UA

VR

NEU

RO

LESI

UB

2

LSH

TM

AU

H-A

S

AZ

UN

OTT

UN

IMIB

AR

S

PHA

RM

O

PED

IAN

ET

USC

TO

TA

L R

P3

TOTA

L Pl

anne

d ef

forts

who

le

proj

ect

% o

f act

ual

effo

rts p

er W

P

WP1 4.1 0.6 1.5 0.8 0.0 0.8 0.0 1.5 0.0 0.6 0.5 0.5 0.0 0.0 0.0 10.9 40.1 27%WP2 3.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.2 0.0 0.7 0.3 0.0 0.0 4.6 40.4 12%WP3 10.3 0.0 0.0 0.0 0.0 0.0 6.0 8.5 0.0 1.2 14.4 6.0 4.6 12.5 0.0 63.5 239.6 27%WP4 9.9 0.0 13.4 0.0 0.0 0.0 2.0 0.0 4.5 0.0 0.0 0.0 0.0 0.0 3.1 32.9 160.1 21%WP5 28.1 0.0 4.0 20.7 0.0 1.8 0.2 0.0 9.0 0.2 0.0 0.0 0.0 0.0 0.0 64.0 120.0 53%WP6 9.3 0.0 0.0 1.3 0.0 2.8 6.0 5.0 0.0 0.2 2.0 4.5 1.3 5.8 4.0 42.2 105.1 40%WP7 2.0 2.9 0.9 0.4 0.0 0.9 0.4 0.9 2.0 0.2 0.5 0.1 0.4 0.0 0.3 11.9 38.9 30%WP8 3.8 5.6 1.0 1.0 0.0 1.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12.5 56.2 22%

TOTAL RP3 70.5 9.1 20.8 24.2 0.0 7.4 15.0 15.9 15.5 2.6 17.4 11.8 6.6 18.3 7.4 242.5

0 30%

TOTAL Planned efforts whole project

209.6 59.3 87 48.6 8 40.5 31.4 33.5 64.5 46.3 34.3 34.3 39.1 39.1 24.9 800.4

% actual efforts per partner

34% 15% 24% 50% 0% 18% 48% 47% 24% 6% 51% 34% 17% 47% 30%


3 Deliverables and milestones tables

TABLE 2. DELIVERABLES

Del. no.

Deliverable name Version WP no.

Lead beneficiary

Nature Disseminationlevel

Delivery date from Annex I

(proj month)

Actual / Forecast

delivery date

Dd/mm/yyyy

StatusNo

submitted/

Submitted

ContractualYes/No

Comments

1.1 Report on Ethical Issues Relevant for the Project V1.5 1 EMC R PU M3 27/07/2008 Submitted Yes

8.1 Project Handbook V1.5 8 FIMIM R CO M3 27/06/2008 Submitted Yes

2.1 List of Events to be monitored V1.6 2 UB2 R PU M4 04/09/2008 Submitted Yes

Updated version

submitted (V1.7)

2.2 Two Validation Sets with Supplementary Information V1.5 2 UB2 R PU M4 16/10/2008 Submitted Yes

Updated version

submitted (V1.6)

1.2 Quality Assurance Guidelines and Procedures V2.0 1 EMC R CO M6 10/09/2008 Submitted Yes

Updated version

submitted (V2.1)

7.1 Communication Plan V1.5 7 FIMIM R CO M6 02/02/2009 Submitted Yes

2.3Medical Event and Drug Terminology Standarisation and Mapping Scheme

V1.2 2 UB2 R PU M9 09/02/2009 Submitted YesUpdated version

submitted (V1.3)

7.2 Report on the EU-ADR communication Tools V1.3 7 FIMIM R PU M12 10/02/2009 Submitted Yes



Del. no.


Lead beneficiary

Nature Dissemination

level


(proj month)

Actual / Forecast

delivery date

Dd/mm/yyyy

Status

No submitted/

Submitted

Contractual

Yes/No

Comments

8.2 Technical and Financial Annual Report#1 V6.7 8 FIMIM R CO M12 15/05/2009 Submitted Yes

Updated version

submitted (V6.8)

4.1Report on the Mining of Pharmacological Databases and Repositories

V1.6 4 UPF R PU M15 18/06/2009 Submitted Yes

1.3 Interim Assessment of the Project V1.2 1 EMC R CO M21 30/03/2010 Submitted Yes

3.1

Description of the Common Data Framework and Software for Local Data Extraction

V2.0 3 EMC R,P PU M21 22/03/2010 Submitted Yes

4.2 Description of Ligand-Based Approaches V1.1 4 UPF R,P PU M21 30/03/2010 Submitted Yes

5.1 Interim Report on Evidence Combination V1.2 5 UAVR R PU M21 30/03/2010 Submitted Yes

7.3 Intermediate Report on Dissemination Activities V1.6 7 FIMIM R PU M24 30/03/2010 Submitted Yes

5.2 EU-ADR System Software V2.1 5 UAVR P CO M24 30/03/2010 Submitted Yes

8.2 Technical and Financial Annual Report#2 V12 8 FIMIM R CO M24 11/06/2010 Submitted Yes

3.2Description of the Text Mining Algorithms and Text Mining Software

V1.4 3 EMC R,P CO M25 26/08/2010 Submitted Yes

4.3 Report on Ligand-Target Simulations V1.4 4 UPF R CO M25 22/09/2010 Submitted Yes



Del. no.


Lead beneficiary

Nature Dissemination

level


(proj month)

Actual / Forecast

delivery date

Dd/mm/yyyy

Status

No submitted/

Submitted

Contractual

Yes/No

Comments

6.1 Interim Report on Retrospective Validation V1.3 6 EMC R RE M27 26/08/2010 Submitted Yes

4.4 Report on Literature and DB Mining V1.5 4 UPF R,P RE M34 20/12/2010 Submitted Yes

6.2 Interim Report on Prospective Validation V1.2 6 EMC R CO M35 05/01/2011 Submitted Yes

4.5Report on Drug-Target-Pathway-Adverse Event Mapping

V1.2 4 UPF R,P CO M36 11/02/2011 Submitted Yes

5.3 Final Report on Evidence Combination V1.0 5 UAVR R PU M36 11/02/2011 Submitted Yes

3.3

Description of the Data Mining Algorithms and Data Mining Software for Local Signal Generation

V1.2 3 EMC R,P CO M42 20/09/2011 Submitted Yes

5.4 Final Version of the EU-ADR System Software V3.2 5 UAVR P CO M42 07/10/2011 Submitted Yes

6.3 Final Report on Retrospective Validation V2.0 6 EMC R PU M42 20/09/2011 Submitted Yes


TABLE 3. MILESTONES

Milestoneno. Milestone name Work

package no Lead beneficiaryDelivery date from Annex Idd/mm/yyyy

AchievedYes/No

Actual / Forecast achievement date

dd/mm/yyyyComments

1 Definition of event list

2 UB2 31/05/2008 Yes 04/09/2008 D2.1

2 Completion of validation sets

2 UB2 31/05/2008 Yes 16/10/2008 D2.2

3 Finalisation of standardisation and mapping terminologies

2 UB2 31/10/2008 Yes 09/02/2009 D2.3

4 Completion of 1st versions of software and algorithms for data extraction and mining of databases and repositories

3, 4 EMC, UPF 30/04/2009 Yes 30/03/2010 Basic functionality of prototype software and algorithms for mining of clinical and biomedical dbs and repositories demonstrated, and reported as D3.1, D4.1, D4.2 and D5.2.

5 Completion of mid-term assessment of the project

1 EMC 31/10/2009 Yes 30/03/2010 Project explicitly evaluated as regards to fulfilment of its objectives, as reported in D1.3

6 Completion of EU-ADR system software version 1, including underlying software components

3,4,5 EMC,UPF,UAVR

31/03/2010 Yes 22/09/2010 Prototype of EU-ADR web system accessible and running flawlessly with basic functionality, reported as D5.2, including completed underlying software components, documented and reported as D3.1, D3.2, D4.2 and D4.3


TABLE 3. MILESTONES

Milestoneno. Milestone name Work

package no Lead beneficiaryDelivery date from Annex Idd/mm/yyyy

AchievedYes/No

Actual / Forecast achievement date

dd/mm/yyyyComments

7 Finalisation of an evidence combination framework

5 UAVR 31/01/2011 Yes 11/02/2011 Combination framework documented and reported as D5.3

8 Completion of final version of the EU-ADR System software, including underlying software components and algorithms

3,4,5 EMC,UPF,

UAVR

31/07/2011 Yes 07/10/2011 Prototype of final EU-ADR web system running flawlessly with full functionality, reported as D5.4, including completed underlying software components and algorithms for data and literature mining, pathway mapping, etc. documented and reported as D3.3, D4.4 and D4.5

9 Completion of retrospective validation studies

6 EMC 31/07/2011 Yes 20/09/2011 Results from retrospective validation documented and reported as D6.3


4 Explanation of the use of the resources

Below it is provided an explanation of personnel costs, subcontracting and any major direct costs incurred by each beneficiary, such as the purchase of important equipment, travel costs, large consumable items, etc. linking them to work packages.

These are listed in the following tables:TABLE 4.1 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY EMC

FOR THE PERIOD 01/02/2010-31/07/2011 Work

PackageItem description Amount in € with

2 decimalsExplanations

1,3,4,5,6,7,8 Personnel direct costs 437,170.00€ Salaries of 8 Researchers for a total of 70.5 PM:J van der Lei (3.5 pm), J Kors (4.1 pm), M. Sturkenboom (3.4 pm), E van Mulligen (2.0 pm) M Schuemie (15.9 pm), P Coloma (19.5 pm), G. Trifiro (9.5), B. Mosseveld (9.1)Salaries of 2 financial management staff for a total of 3.5 pm: T de Ben (1.7 pm), A Woerdeman (1.8 pm)

3,6 Subcontracting 65,200.00€ Subcontractor SIMG1,3,4,5,6,7 Other direct costs 48,301.94€ Travel costs:

- 7th Consortium Meeting (CM), Sitges, February 15-16, 2010, E van Mulligen, J van der Lei, M Schuemie, M Sturkenboom, P Coloma, J Kors, G Trifiró, S Romio,

- 2nd Project Review, Brussels April 13-14, 2010, E van Mulligen, J van der Lei, M Schuemie, M Sturkenboom,J Kors, G Trifiró,

- 8th CM, Rotterdam, May 25-26, 2010, E van Mulligen, J van der Lei, M Schuemie, M Sturkenboom, P Coloma, J Kors, G Trifiró, S Romio, T de Ben

- 9th CM, Florence, October 25-26, 2010, E van Mulligen, J van der Lei, M Schuemie, M Sturkenboom, P Coloma, J Kors, G Trifiró, S Romio, V Patadia,

- 10th CM, Aveiro, February 2-3, 2011, E van Mulligen, J van der Lei, M Schuemie, M Sturkenboom, P Coloma, J Kors, G Trifiró,

- 11th CM, Barcelona, April 4-5, 2011, E van Mulligen, J van der Lei, M Schuemie, M Sturkenboom, P Coloma, G Trifiró, B Singh

- 12th CM, Barcelona, June 22-23, 2011, E van Mulligen, J van der Lei, M Schuemie, M Sturkenboom, P Coloma, G Trifiró, J Kors, B Singh

- Other travel costs and travel costs related to dissemination activities, including the Conferences (March 2010, September 2010, January 2011) attended by M Schuemie, the Conference July 2010 attended by G. Trifiro, the Conference (July 2010) attended by P. Coloma and the Conferences (April 2010, June 2010, Oktober 2010, December 2010, January 2011) attended by M. Sturkenboom

1,3,4,5,6,7 Remaining direct costs 4,184.06€ Various cost: mailing, teleconferencing, tuition (P. Coloma, Training program Clinical Epidemiology) and hardware costs. Indirect costs 293,793.00€ 60% rate for indirect costs

TOTAL COSTS 848,649.00€

TABLE 4.2 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY FIMIM


FOR THE PERIOD 01/02/2010-31/07/2011 Work

PackageItem description Amount in € with

2 decimalsExplanations

1,7,8 Personnel direct costs 41,169.00€ Salaries of Carlos Díaz (2.5 p/m), Eva Molero (2.9 p/m), Sandra Pla (3.7 p/m)7 Subcontracting 220.00€ Website Maintenance carried out by the subcontractor “Designatius.com”

7,8 Travel costs and meeting organisation

22,125.00€ - 7th Consortium Meeting (CM), Sitges, February 15-16, 2010. Meeting organised and hosted by FIMIM. The organisation costs include the dinner, catering and hiring meeting rooms (4,673.79€).

- 2nd Project Review, Brussels April 13-14, 2010, C Diaz, E Molero, S Pla- 8th CM, Rotterdam, May 25-26, 2010, C Diaz, E Molero,- 9th CM, Florence, October 25-26, 2010, C Diaz, E Molero, S Pla- 10th CM, Aveiro, February 2-3, 2011, C Diaz, E Molero, S Pla- 11th CM, Barcelona, April 4-5, 2011. Meeting organised and hosted by FIMIM. The organisation costs include the

dinner, catering and hiring meeting room (4,416.54€). - 12th CM, Barcelona, June 22-23, 2011. Meeting organised and hosted by FIMIM. The organisation costs include the

dinner, catering and hiring meeting room (2,528.85€).

Travel costs for Medinfo Conference (September, 2010) attended by Eva Molero 7,8 Remaining direct costs 893.00€ Print of Flyer and courier costs.

Indirect costs 38,511.00€ 60% rate for indirect costsTOTAL COSTS 102,918.00€

TABLE 4.3 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY UPFFOR THE PERIOD 01/02/2010-31/07/2011

Work Package

Item description Amount in € with 2 decimals

Explanations

1,4,5,7,8 Personnel direct costs 56,842.47€ Salaries of own personnel, 9.81 p/m,(2 senior scientists and 1 pre-doc) and new personnel for the project, 11 p/m, (1 pre-doctoral student).

4,8 Subcontracting 7,389.00€ Subcontracting of Work performed by TAU, and Audit Certificates.1,4,5 Travel costs 7,623.27€ Travel costs for Project meetings:

- 7th Consortium Meeting (CM), Sitges, February 15-16, 2010, L Furlong, A Bauer-Muhren, F Sanz- 2nd Project Review, Brussels April 13-14, 2010, L Furlong, J Mestres, F Sanz- 8th CM, Rotterdam, May 25-26, 2010, L Furlong- 26th. ICPE 2010 Project dissemination, Brighton, England. August 19-22, 2010 L Furlong- ECCB10 Project dissemination, Ghent, Belgium. September 26, 2010, A Bauer-Mehren- Collaboration Meeting, Paris, France. September 28, 2010, A Bauer-Mehren, L Furlong, - ICSB2010 Project dissemination, Edinburgh, England. October 11-14, 2010, A Bauer-Mehren, - 9th CM, Florence, October 25-26, 2010, L Furlong, A Bauer-Muhren, F Sanz, J Mestres- 10th CM, Aveiro, February 2-3, 2011, A Bauer-Mehren, L Furlong

Remaining direct costs 5,563.04€ Scientific collaborations about “Mapping side effects on drug-target networks”, this collaborations were specific contributions from visitor experts.


Congress fees for Laura Furlong (ICPE2010) and Anna Bauer-Mehren (ECCB2010).Indirect costs 42,017.27€ 60% rate for indirect costs


TABLE 4.4 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY UAVRFOR THE PERIOD 01/02/2010-31/07/2011

Work Package


Explanations

1,5,6,7,8 Personnel direct costs 133,741.94€ Salaries corresponding to 24.20 p/m:- José Luís Oliveira: 6.35 p/m- José Pinto: 8.5 p/m- Carlos Costa: 4.25 p/m- Joaquim Arnaldo Martins: 5.1 p/m

1,5,6,7,8 Other direct costs 12,977.39€ General meetings of the Project and several joint activities, namely:- 7th Consortium Meeting (CM), Sitges, February 15-16, 2010, Jose Luis Oliveira- 2nd Project Review, Brussels April 13-14, 2010, Jose Luis Oliveira,- 8th CM, Rotterdam, May 25-26, 2010, Jose Luis Oliveira- 9th CM, Florence, October 25-26, 2010, Jose Luis Oliveira- 10th CM, Aveiro, February 2-3, 2011, Local Organisation. - 11th CM, Barcelona, April 4-5, 2011, Jose Luis Oliveira- 12th CM, Barcelona, June 22-23, 2011, Jose Luis Oliveira

Presentation of Project results in international conferences (PACBB2010, ITAB2010).Indirect costs 88,031.6€ 60% rate for indirect costs


TABLE 4.5 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY UB2FOR THE PERIOD 01/02/2010-31/07/2011


Work Package


Explanations

1,5,6,7,8 Personnel direct costs 44,510.89 € Salaries for partial workload of 7 researchers for the period according to time sheets and as detailed below:N. Moore: 1.3 p/m, A Fourrier-Réglat: 3.9 p/m, A Pariente: 0.8 p/m, F. Salvo: 0.5 p/m, P Avillach: 0.3 p/m, F. Mougin: 0.03 p/m, G Diallo: 0.6 p/m

5,6 Other direct costs 17,102.73 € Travel costs for Project meetings:- 7th Consortium Meeting (CM), Sitges, February 15-16, 2010, A. Fourrier-Réglat, F. Mougin, P. Avillach, G. Diallo- 2nd Project Review, Brussels April 13-14, 2010, A. Fourrier-Réglat, F. Thiessard- 8th CM, Rotterdam, May 25-26, 2010, P. Avillach, G. Diallo- 9th CM, Florence, October 25-26, 2010, A. Fourrier-Réglat, P. Avillach, G. Diallo- 10th CM, Aveiro, February 2-3, 2011, G. Diallo- 11th CM, Barcelona, April 4-5, 2011, A. Fourrier-Réglat, G. Diallo

Travel costs for Medinfo Conference (September, 2010) attended by P AvillachIndirect costs 36,968.17€ 60% rate for indirect costs


TABLE 4.6 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY LSHTMFOR THE PERIOD 01/02/2010-31/07/2011

Work Package


Explanations

2,3,4,5,6,7 Personnel direct costs 73,315.43€ Contribution to salaries of one computer programmer (Ferran Orsola 0.11 p/m), one data manager (Richard Jackson 0.9 p/m); one Project co-ordinator (Ruqayya Suleman 6.6 p/m), one statistician (David Prieto 0.4 p/m ), one epidemiologist (Justin Matthews 7 p/m ) Total 15 p/m

2,3,4,5,6 Other direct costs 6,141.36€ Travel costs for Project meetings:- 7th Consortium Meeting (CM), Sitges, February 15-16, 2010, Mariam Molokhia- 2nd Project Review, Brussels April 13-14, 2010, Mariam Molokhia- 8th CM, Rotterdam, May 25-26, 2010, Mariam Molokhia, Justin Matthews- ICPE conference Brighton, UK Aug 2010, David Prieto- 9th CM, Florence, October 25-26, 2010, Mariam Molokhia- 11th CM, Barcelona, April 4-5, 2011, Mariam Molokhia, Justin Matthews, David Prieto- 12th CM, Barcelona, June 22-23, 2011, Mariam Molokhia, Justin Matthews

Indirect Costs 47,674.07€ 60% rate for indirect costs TOTAL COSTS 127,130.86€

TABLE 4.7 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY AUH-AS FOR THE PERIOD 01/02/2010-31/07/2011


Work Package


Explanations

1 Personnel direct costs 8,926.70€ Salaries for one admin person for 1 month and for one statistician for 0.5 month3 Personnel direct costs 18,124.32€ Salaries for two statisticians for 1 month and 3 months, respectively3 Personnel direct costs 20,461.96€ Salaries for two statisticians for 3.7 months and 0.8 months, respectively6 Personnel direct costs 18,407.23€ Salaries for two statisticians for 1 month and 3 months, respectively 6 Personnel direct costs 4,764.37€ Salary for one statistician for 1 month7 Personnel direct costs 5,341.48€ Salaries for one statistician for 0.3 month and for one admin person for 0.6 month

1,3,6 Travel costs 4,280.91€ Costs for one person attending:- 8th Consortium Meeting (CM) Rotterdam, May 25-26, 2010- 9th CM, Florence, October 25-26, 2010- 10th CM, Aveiro, February 2-3, 2011- 11th CM, Barcelona, April 4-5, 2011

Indirect costs 48,184.18€ 60% rate for indirect costs TOTAL COSTS 128,491.14€

TABLE 4.8 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY AZ FOR THE PERIOD 01/02/2010-31/07/2011

Work Package


Explanations

4,5,7 Personnel direct costs 106,431.23€ Salaries for a post doc 9 months (Ernst Ahlberg). A researcher for 2.5 months (Samuel Andersson). PA for 2 months (Marita Franzén). Senior researcher for 2 months (Scott Boyer)

4,5,7 Travel costs 22,825.90€ Travel costs for Project meetings:- 7th Consortium Meeting (CM), Sitges, February 15-16, 2010, Scott Boyer, Samuel Andersson- 8th CM, Rotterdam, May 25-26, 2010, Scott Boyer, Samuel Andersson, Ernst Ahlberg- 9th CM, Florence, October 25-26, 2010, Scott Boyer, Ernst Ahlberg - 10th CM, Aveiro, February 2-3, 2011, Scott Boyer, Ernst Ahlberg- 11th CM, Barcelona, April 4-5, 2011, Scott Boyer, Ernst Ahlberg- 12th CM, Barcelona, June 22-23, 2011, Scott Boyer, Ernst Ahlberg- Additional travel costs for 3 conferences attended by Scott Boyer and one attended by Ernst Ahlberg.

Indirect costs 38,777.14€ Actual indirect costs TOTAL COSTS 168,034.27€

TABLE 4.9 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY UNOTT FOR THE PERIOD 01/02/2010-31/07/2011


Work Package


Explanations

1,2,3,5,6,7 Personnel direct costs 29,722.00€ Salary for Professor Julia Hippisley-Cox 1.2.10 - 31.7.11

1 Personnel direct costs 751.00€ April McCambridge-general administrative tasks including completion of forms, travel arrangements and timesheets1 Personnel direct costs 53.00€ Project Admin Support-Jill Harris (Accounting, Research Innovation Services)

1,2,3,5,6 Travel costs 2,075.00€ Travel costs for Project meetings:- 2nd Project Review, Brussels 13-14, April 2010, Julia Hippisley-Cox- 8th CM, Rotterdam, May 25-26, 2010, Julia Hippisley-Cox- 11th CM, Barcelona, April 4-5, 2011, Julia Hippisley-Cox

3,7 Remaining direct costs 58,695.00€ QResearch data access:- April 10 extract costs for WP3 €11,792.45- July 10 extract costs for WP3 €11,792.45- April 11 Facilities cost transfer for WP3 €11,461.32- Extract costs Gold run for WP3 €11,792.45- Extract costs Silver run for WP3 €11,792.45Printing of poster for display at University of Nottingham €63.88 June 2010 WP7


TABLE 4.10 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY UNIMIB FOR THE PERIOD 01/02/2010-31/07/2011

Work Package


Explanations

1,3,6,7 Personnel direct costs 15,334.03€ Salaries of ordinary staff: 1 professor for 0.5 p/m, 2 researchers for 1.5 p/m, 1 associate professor for 1 p/m3 Personnel direct costs 39,394.42€ Salaries of temporary staff: 1 stat-info technician for 14.4 p/m

1,3,6 Travel costs 2,099.61€ Travel costs for Project meetings:- 7th Consortium Meeting (CM), Sitges, February 15-16, 2010, Lorenza Scotti- 8th CM, Rotterdam, May 25-26, 2010, Lorenza Scotti- 9th CM, Florence, October 25-26, 2010, Lorenza Scotti, Arianna Ghirardi- 10th CM, Aveiro, February 2-3, 2011, Lorenza Scotti, Arianna Ghirardi- 11th CM, Barcelona, April 4-5, 2011, Lorenza Scotti, Arianna Ghirardi

1,3,6 Remaining direct costs 807.51€ Pc, printer, monitor, memory unit DepreciationIndirect costs 34,581.64€ 60% rate for indirect costs


TABLE 4.11 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY ARS FOR THE PERIOD 01/02/2010-31/07/2011

Work Item description Amount in € with Explanations


Package 2 decimals1,2,3,6,7 Personnel direct costs 39,072.67€ Salaries of 4 researches: 3 p/m, 5 p/m, 3.2 p/m and 0.6 p/m

6 Subcontracting 440.80€ Chart review from collaborating hospital 1,2,3,6 Travel costs 5,057.20€ Travel costs for Project meetings (1,829.54€):

- 7th Consortium Meeting (CM), Sitge February 14, 2010, Rosa Gini;- 10th CM, Aveiro, February 2-6, 2011, Rosa Gini; Franccesco Innocenti- 11th CM, Barcelona, April 4-5, 2011, Rosa Gini;- 12th CM, Barcelona, June 22-23, 2011, Rosa Gini;

Consortium meetings (3,227.66€):9th CM, Florence, October 25-26, 2010

7 Travel costs 3,243.25€ Travel costs for Medinfo Conference (September, 2010) attended by Rosa Gini Indirect costs 28,423.87€ 60% rate for indirect costs


TABLE 4.12 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY PHARMO FOR THE PERIOD 01/02/2010-31/07/2011

Work Package


Explanations

2,3,6,7 Personnel direct costs 56,113.00€ Salaries of statistician, scientific director, software engineer, managing director, data logistic manager for a total 6.6 p/m2,3,6 Other direct costs 7,081.70€ Data costs: 3.000,00.€

Travel costs for Project meetings (4.081,70€):- 7th Consortium meeting (CM), Sitges February 14, 2010. 2 travelers (Huub M.P.M. Straatman, Ron Herings)- 9th CM Florence, October 25-26, 2010. 2 travelers (Huub M.P.M. Straatman, Ron Herings)- 10th CM, Aveiro, February 2-6, 201. 2 travelers (Huub M.P.M. Straatman, Ron Herings)- 11th CM, Barcelona, April 4-5, 2011. 1 traveler (Ron Herings)- 12th CM, Barcelona, June 22-23, 2011. 1 traveler (Ron Herings)


TABLE 4.13 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY PEDIANET FOR THE PERIOD 01/02/2010-31/07/2011

Work Package


Explanations


3,6 Personnel direct costs 37,802.66€ Salary of Data Manager (15.3 p/m)3,6 Subcontracting 15,000.00€ These costs were incurred for the payment to the International Pharmaco-epidemiology and Pharmaco-economics Research

Center (IPPRC) for the elaboration of large databases and the statistical analysis of epidemiological studies (3 p/m) 3 Other direct costs 4,219.76€ Purchase of personal computers and of a server for data collection and management (depreciation costs only are charged)


TABLE 4.14 PERSONNEL, SUBCONTRACTING AND OTHER MAJOR COST ITEMS FOR BENEFICIARY USC FOR THE PERIOD 01/02/2010-31/07/2011

Work Package


Explanations

4,6 Personnel direct costs 25,985.00€ Salaries of Prof. M. Isabel Loza (1.35 p/m), José Manuel Brea (3.02 p/m), Prof. María Angeles Castro (2.04 p/m), José Manuel Santamaría (0.69 p/m)

7 Personnel direct costs 1,338.00€ Salary of Prof. M. Isabel Loza (0.30 p/m)4,6 Travel costs 5,902.00€ Travel costs for Project meetings:

- 7th Consortium Meeting (CM), Sitges, February 15-16, 2010, Mabel Loza, José Manuel Brea- 8th CM, Rotterdam, May 25-26, 2010, Mabel Loza, Ainhoa Nieto- 9th CM, Florence, October 25-26, 2010, Mabel Loza, Ainhoa Nieto- 10th CM, Aveiro, February 2-3, 2011, Mabel Loza, Ainhoa Nieto, José Manuel Santamaría- 11th CM, Barcelona, April 4-5, 2011, Ainhoa Nieto



5 Financial statements – Form C and Summary financial reportPlease see PDF file with all the Form Cs and the summary of costs reported.

6 Certificates

Beneficiary Organisation short name

Certificate on the financial statements provided?yes / no

Any useful comment, in particular if a certificate is not provided

1 EMC Yes Expenditure threshold reached2 FIMIM No Expenditure threshold not reached3 UPF Yes Expenditure threshold reached4 UAVR No Expenditure threshold not reached6 UB2 No Expenditure threshold not reached7 LSHTM No Expenditure threshold not reached8 AUH-AS No Expenditure threshold not reached9 AZ No Expenditure threshold not reached10 UNOTT No Expenditure threshold not reached11 UNIMIB No Expenditure threshold not reached12 ARS No Expenditure threshold not reached13 PHARMO No Expenditure threshold not reached14 PEDIANET No Expenditure threshold not reached15 USC No Expenditure threshold not reached


7 Annexes – Current Top Risks

RISK DOCUMENTATION FORM

Risk ID Resolved:

TYPE OF RISK: Threat Opportunity CLASSIFICATION: Technical - SoftwareWORK PACKAGE/ACTIVITY: WP3RISK OWNER12: Martijn Schuemie (EMC)LAST UPDATE: 30/09/2011

DESCRIPTION:Risk located in the performance of the Jerboa software, as it is difficult to predict the actual performance of Jerboa on large datasets in various environments. However, the work in WP3 has been mostly caried out without major problems, and data extraction is undertaken off-line (outside the EU-ADR Web Platform), so impact on the project is regarded as low. However, this is a risk currently perceived as more related to the post-project phase if many databases participate and more dynamic data turnover is essential. Therefore, it is an important risk for the future, so overall Impact is deemed to be Medium.

Proximity in time: Medium term (3 to 12 months)

Impact on project: 2-Medium (a)

Probability of happening: 2-Medium (b)

Exposure: 4 (a)*(b)

Mitigation Approaches13

Use of Jerboa in different environments (e.g. benchmarking exercises with Asian and American databases) and other projects (e.g. SOS, ARITMO, VAESCO, SAFEGUARD) with as realistic datasets as possible.

Trigger EventsPerformance problems in Jerboa when used in large datasets (errors, processing delays, etc.)

Contingency Plans14

Initiate and resource a technical workgroup dedicated to analyse performance issues and ways to solve them.

12 Partner in the best position to recommend mitigation strategies, develop contingency plans and monitor the status of the risk.13 All threat risks with medium or high probability or impact should have a mitigation strategy.14 All risks having an exposure equal or greater than 4 should have a contingency plan in advance.

© Copyright 2011 EU-ADR Consortium

01-013

56


Risk ID Resolved:

TYPE OF RISK: Threat Opportunity CLASSIFICATION: ExploitationWORK PACKAGE/ACTIVITY: WP7RISK OWNER: Eva Molero (FIMIM)LAST UPDATE: 30/09/2011

DESCRIPTION: Lack of consensus regarding use of results after project jeopardise post-project phase and adequate exploitation of results. FORMTEXT Partners are focused on the technical scientific work of the project; therefore, they may postpone the decisions on exploitation until the end of the project, where agreements will be more difficult.

Proximity in time: Short term (<3 months)

Impact on project: 3-High (a)

Probability of happening: 1-Low (b)

Exposure: 3 (a)*(b)

Mitigation ApproachesTo arrange brainstorming sessions within the consortium in order to know partners' views regarding the results after project life (DONE). To ask partners (through a survey form) their opinion on the implications of the post-project phase (DONE). Analyse lightweight approaches to facilitate initial exploitation (DONE)

Trigger EventsLack of appropriate feedback derived from the mitigation activities above.

Contingency Plans Reach agreement with selected partners, look for consensus and IPR arrangements to ensure continuation of activities with the involvement of motivated partners only.


05-0003

57


Risk ID Resolved:

TYPE OF RISK: Threat Opportunity CLASSIFICATION: ExploitationWORK PACKAGE/ACTIVITY: WP3RISK OWNER: Johan van der Lei (EMC)LAST UPDATE: 07/09/2011

DESCRIPTION: Difficulties to prove superiority over traditional methods. EU-ADR intended to serve to identify signals better and faster than spontaneous reporting systems. However, comparison is difficult.

Proximity in time: Short term (<3 months)

Impact on project: 3-High (a)

Probability of happening: 1-Low (b)

Exposure: 3 (a)*(b)

Mitigation Approaches Focus on validation activities and all WPs to support them in the final months of the project.

Trigger Events Inconclusive results obtained in validation deliverables.

Contingency Plans Reinforce communication activities addressed to present EU-ADR as complementary (i.e. not a replacement) for SRS, emphasizing areas in which it performs better.


05-0004

58

front page - european commission : cordis :...

Documents