Scholars@DrupalIntegrating Scholars@Duke

with Drupal - a case study

Ryan Scotton, Developer

Coalmarch Productions

Coalmarch & DukeDuke Magazine

Duke Gardens

Coalmarch & DukeDepartment of Ophthalmology

Department of Medicine

School of Medicine

Private Diagnostic Clinic

Department of Neurobiology

The SOM Prototype

So, what’s this Drupal thing again?

● Extremely powerful, highly customizable CMS framework

● Uses a PHP/MySQL backend● Powered by ‘modules’ of functionality - core,

contributed, and, in this case, custom

Duke Department of Medicine’s Goals

● Be able to quickly add/update individual scholar profiles to the website

● Create batch functionality to pull in all relevant profiles and associated data tied to DOM

● Set up regular automated syncs with Scholars@Duke server to keep info up-to-date

Interfacing with Scholars@Duke

● Widgets data was more than enough to gather all necessary info

● Used ‘complete’ widgets data in JSON format (minimizes server requests by grabbing everything at once)

Widget Builder -> Feed URLThe widget builder gives an example of the type of URL structure we’ll be working with to pull down the data.

Swap ‘publications’ token with ‘complete’, the .js extension with .json, remove the unnecessary _GET parameters, and we have what we need!

First, pull down the data and convert JSON into associative array

$json_text = file_get_contents('https://scholars.duke.edu/widgets/api/v0.9/people/complete/all.json?uri=https://scholars.duke.edu/individual/' . $id);

$json_array = json_decode($json_text, TRUE);

then...

Voila!Now, we have all the information we need in a very readable and accessible associative array

Setting up a single scholar entry form in Drupal

Setting up a single scholar entry form in Drupal

Let's take a look under the hood

Ok, but how do we pull in all of DOM’s profiles (1500+) at once?

● Organization ID (DOM = org50000844)● Example:

○ DOM’s organization feed○ Handy dandy JSON viewer

Setting up recurring Scholars@Duke syncs using

Drupal's cron scheduler

Why sync only once per week?● DOM has 1541 profiles currently, with (formerly) non

web-optimized photos and potentially hundreds of publications per profile

● Each Scholars sync = gigabytes of data (>2.6gb in images alone, originally)

● Solution: limit full sync to weekly run off peak-hours, optimize images to reduce load

Wait, what’s the ‘queue’ business?● Drupal’s cron scheduler times out at 240 seconds● Dev site was going haywire attempting to sync using

cron (timeouts, attempting to re-run cron while it’s already running, etc)

● Enter Drupal Queue - integrates with Cron via hook_cron_queue_info() and handles long-running background tasks, taking processing burden off of Cron

Enough about logistics: how did you actually use Scholars data?

● Three Content Types:○ Profile○ Student/Trainee○ Publication

● Each content type shares a ‘scholar_id’ field - primary key used to avoid redundancy and update the proper node

● ‘vivoType’ field used to distinguish profiles from student/trainees● Set up pointers (aka node references) from profiles/students/trainees to

publications

Enough about logistics: how did you actually use Scholars data? (cont.)

● Views module uses contextual filter (current node ID) to pull individual’s publications

● Used PHP’s cURL library to import profile images - images overwritten on each sync

● Focal Point module allows for easily assigning profile cropping settings● Used organizationUri of primary position attached to each Scholar to tag

appropriate division/program - used for filtering and site accessibility

Default focal point, set by custom module

Before After

Division tag allows for quick access to division homepage

FIN. Here are some tools & resources to set you on your way!

● Drupal’s Devel module (dsm() is your best friend!)○ https://www.drupal.org/project/devel

● Web-based JSON parser/viewer○ http://jsonviewer.stack.hu/

● The Focal Point module, used for image styles scaling/cropping○ https://www.drupal.org/project/focal_point

● Setting up a basic ‘Add Scholar’ form○ https://gist.github.com/rscotton/a4a8678182dd244a0f2b

● Setting up a recurring batch tied to Cron○ https://gist.github.com/rscotton/a542f843ea2e80503424

● Coalmarch’s write-up on Drupal Queue○ http://www.coalmarch.com/blog/drupal-queue-a-crons-best-batch-processing-friend

Any Questions?Follow up with me at

rscotton@coalmarch.com

or on LinkedIn

https://www.linkedin.com/in/rscotton

You can find this presentation here: http://bit.ly/1IetD39