rebecca pankow supervisor: veronique lefebure cern summer student program 2010 updating the syntax...
TRANSCRIPT
![Page 1: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/1.jpg)
Rebecca PankowSupervisor: Veronique LefebureCERN Summer Student Program
2010
Updating the Syntax ofPAN Template Files
![Page 2: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/2.jpg)
Introduction
The CF-ASI section provides infrastructure and support for the QUATTOR framework used for installing, configuring and managing the 8000
machines of the computer center.
PAN is a functional programming language that translates a high-level site configuration written by a system administrator in the pan configuration language to a machine-readable representation.
![Page 3: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/3.jpg)
Introduction
object template hello;'/message' = 'hello';
<?xml version="1.0" encoding="UTF-8"?><nlist format="pan" name="profile"><string name="message">hello</string></nlist>
![Page 4: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/4.jpg)
Introduction
object template nfsserver.example.org; '/hardware/location/rack' = 'IBM04'; '/hardware/location/slot' = 25; '/hardware/ram' = 2048; '/hardware/cpu/model' = 'Intel Xeon'; '/hardware/cpu/speed' = 2.5; '/hardware/cpu/arch' = 'x86_64'; '/hardware/cpu/cores' = 4; '/hardware/cpu/number' = 2;
![Page 5: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/5.jpg)
Project Details
• An upcoming release of the PAN compiler used by QUATTOR changes the PAN programming language syntax
• This requires an update of 30,000 pre-existing PAN configuration files.
• Need to search for the following within each file and make appropriate change:– include x; include { ‘x’ };– type “x” = … bind “x” = …
![Page 6: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/6.jpg)
Updating the Syntax
• Originally tried to update the syntax by parsing the files (faster to run)
• Final solution uses regular expressions for more flexibility (slower to run)if ($curr_line =~ /^\s*include\s+([\w\/-]+)\s*;/) { $add_line = "include {'$1'};\n"; } elsif ($curr_line =~ /^\s*type\s+["']([\w\/-]+)["']\s*=\s*([\
w\/-]+)\s*;/) { $add_line = "bind '$1' = $2;\n";
}
![Page 7: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/7.jpg)
Verifying Solution Correctness
• The code in each template file generates an XML file
• If the regular expressions are correct, there should be no difference between the XML files generated by the updated template files and the XML files generated by the original template files
![Page 8: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/8.jpg)
Verifying Solution Correctness
• Wrote a small script to loop through all the files and run a diff between the original XML files and the XML files generated by the updated template files
• Updated template files now generate identical XML files as compared to the original XML files
![Page 9: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/9.jpg)
swrep-soap-client archive
• The swrep-soap-client should have a way to tell whether or not a rpm is archived
• Accomplished by using a predefined function to identify whether an rpm is archived or not
• Appended information on archived status to output of command
![Page 10: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/10.jpg)
Uploading rpms to all platforms
• Allow users to upload a particular RPM to all known platforms inside SWrep, without having to extract them via "listplatforms" and then manually specifying them on the command line
• Client code modified by getting the list of all platforms available and looping the uploading code through the list
![Page 11: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/11.jpg)
Extracting correct substring from differently formatted strings
• Two different servers return different strings when the following command is called:
temp=`cat $TMPFILE | grep URL | head -1 | cut "-d " -f5 | cut '-d"' -f2`• Error due to different string formats in $TMPFILE• Format for lxplus :
– 2010-06-22 18:03:34 URL:https://apex.cern.ch:443/pls/htmldb_itcore/f?p=116:203:6844064748542101::::P203_ID:1005 [23443] -> "f?p=116:203:6844064748542101::::P203_ID:1005.html“
– Returns “->” (incorrect)• Format for lxadm :
– 10:56:12 URL:https://apex.cern.ch:443/pls/htmldb_itcore/f?p=116:203:5808253488607197::::P203_ID:1002 [25146] -> "f?p=116:203:5808253488607197::::P203_ID:1002.html“
– Returns “f?p=116:203:5808253488607197::::P203_ID:1002.html” (correct)• Solution: Change expression to
temp= `cat $TMPFILE | grep URL | sed 's/^.*"f?p/f?p/' | sed 's/".*$//'`
![Page 12: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/12.jpg)
Automatic file deletion
• Add code that will automatically deleted selected files based on time stamp in file name:
• Example file names:
SDBofflinecopy-20100605-0818.zip
SDBofflinecopy-YYYYMMDD-HH24MI.zip
![Page 13: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/13.jpg)
Automatic file deletion
• deletecopy_filename="SDBofflinecopy-`date +%Y%m%d --date '2 month ago'`-`date +%H`*"
old_copy="${full_path}/${deletecopy_filename}"
rm -f $old_copy
• If $full_path already has a '/' at the end then the line should be changed to:
old_copy="${full_path}${deletecopy_filename}"
![Page 14: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/14.jpg)
tpl_viewer.php
• A php script used to generate links to template files does not work with certain template files
• Example line from template file:• $line = "include { if
(exists('customization/'+ELFMS_RESOURCE+'/'+ELFMS_PROJECT+'/'+ELFMS_CUSTOMIZATION+'/config')) 'customization/'+ELFMS_RESOURCE+'/'+ELFMS_PROJECT+'/'+ELFMS_CUSTOMIZATION+'/config' };";
• Links are generated via regular expressions• Previous line does not correctly match any regular
expressions in tpl_viewer.php• Regular expressions will need to be updated to correctly
match all lines
![Page 15: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/15.jpg)
Cultural Experiences
![Page 16: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/16.jpg)
Cultural Experiences
![Page 17: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/17.jpg)
Cultural Experiences
![Page 18: Rebecca Pankow Supervisor: Veronique Lefebure CERN Summer Student Program 2010 Updating the Syntax of PAN Template Files](https://reader035.vdocuments.mx/reader035/viewer/2022062421/56649d975503460f94a8021f/html5/thumbnails/18.jpg)
Acknowledgements
• Veronique Lefebure• Homer Neal• Jean Krisch• Jeremy Herr• CERN• National Science Foundation• University of Michigan