rebecca pankow supervisor: veronique lefebure cern summer student program 2010 updating the syntax...

Rebecca PankowSupervisor: Veronique LefebureCERN Summer Student Program

2010

Updating the Syntax ofPAN Template Files

Introduction

The CF-ASI section provides infrastructure and support for the QUATTOR framework used for installing, configuring and managing the 8000

machines of the computer center.

PAN is a functional programming language that translates a high-level site configuration written by a system administrator in the pan configuration language to a machine-readable representation.

Introduction

object template hello;'/message' = 'hello';

<?xml version="1.0" encoding="UTF-8"?><nlist format="pan" name="profile"><string name="message">hello</string></nlist>

Introduction

object template nfsserver.example.org; '/hardware/location/rack' = 'IBM04'; '/hardware/location/slot' = 25; '/hardware/ram' = 2048; '/hardware/cpu/model' = 'Intel Xeon'; '/hardware/cpu/speed' = 2.5; '/hardware/cpu/arch' = 'x86_64'; '/hardware/cpu/cores' = 4; '/hardware/cpu/number' = 2;

Project Details

• An upcoming release of the PAN compiler used by QUATTOR changes the PAN programming language syntax

• This requires an update of 30,000 pre-existing PAN configuration files.

• Need to search for the following within each file and make appropriate change:– include x; include { ‘x’ };– type “x” = … bind “x” = …

Updating the Syntax

• Originally tried to update the syntax by parsing the files (faster to run)

• Final solution uses regular expressions for more flexibility (slower to run)if ($curr_line =~ /^\s*include\s+([\w\/-]+)\s*;/) { $add_line = "include {'$1'};\n"; } elsif ($curr_line =~ /^\s*type\s+["']([\w\/-]+)["']\s*=\s*([\

w\/-]+)\s*;/) { $add_line = "bind '$1' = $2;\n";

}

Verifying Solution Correctness

• The code in each template file generates an XML file

• If the regular expressions are correct, there should be no difference between the XML files generated by the updated template files and the XML files generated by the original template files

Verifying Solution Correctness

• Wrote a small script to loop through all the files and run a diff between the original XML files and the XML files generated by the updated template files

• Updated template files now generate identical XML files as compared to the original XML files

swrep-soap-client archive

• The swrep-soap-client should have a way to tell whether or not a rpm is archived

• Accomplished by using a predefined function to identify whether an rpm is archived or not

• Appended information on archived status to output of command

Uploading rpms to all platforms

• Allow users to upload a particular RPM to all known platforms inside SWrep, without having to extract them via "listplatforms" and then manually specifying them on the command line

• Client code modified by getting the list of all platforms available and looping the uploading code through the list

Extracting correct substring from differently formatted strings

• Two different servers return different strings when the following command is called:

temp=`cat $TMPFILE | grep URL | head -1 | cut "-d " -f5 | cut '-d"' -f2`• Error due to different string formats in $TMPFILE• Format for lxplus :

– 2010-06-22 18:03:34 URL:https://apex.cern.ch:443/pls/htmldb_itcore/f?p=116:203:6844064748542101::::P203_ID:1005 [23443] -> "f?p=116:203:6844064748542101::::P203_ID:1005.html“

– Returns “->” (incorrect)• Format for lxadm :

– 10:56:12 URL:https://apex.cern.ch:443/pls/htmldb_itcore/f?p=116:203:5808253488607197::::P203_ID:1002 [25146] -> "f?p=116:203:5808253488607197::::P203_ID:1002.html“

– Returns “f?p=116:203:5808253488607197::::P203_ID:1002.html” (correct)• Solution: Change expression to

temp= `cat $TMPFILE | grep URL | sed 's/^.*"f?p/f?p/' | sed 's/".*$//'`

Automatic file deletion

• Add code that will automatically deleted selected files based on time stamp in file name:

• Example file names:

SDBofflinecopy-20100605-0818.zip

SDBofflinecopy-YYYYMMDD-HH24MI.zip

Automatic file deletion

• deletecopy_filename="SDBofflinecopy-`date +%Y%m%d --date '2 month ago'`-`date +%H`*"

old_copy="${full_path}/${deletecopy_filename}"

rm -f $old_copy

• If $full_path already has a '/' at the end then the line should be changed to:

old_copy="${full_path}${deletecopy_filename}"

tpl_viewer.php

• A php script used to generate links to template files does not work with certain template files

• Example line from template file:• $line = "include { if

(exists('customization/'+ELFMS_RESOURCE+'/'+ELFMS_PROJECT+'/'+ELFMS_CUSTOMIZATION+'/config')) 'customization/'+ELFMS_RESOURCE+'/'+ELFMS_PROJECT+'/'+ELFMS_CUSTOMIZATION+'/config' };";

• Links are generated via regular expressions• Previous line does not correctly match any regular

expressions in tpl_viewer.php• Regular expressions will need to be updated to correctly

match all lines

Cultural Experiences

Cultural Experiences

Cultural Experiences

Acknowledgements

• Veronique Lefebure• Homer Neal• Jean Krisch• Jeremy Herr• CERN• National Science Foundation• University of Michigan