nricaribou - lecol-ck.ca filephpmaker (template design and automatic php code generation for the...

$: NRICARIBOU - lecol-ck.ca filePHPMaker (template design and automatic PHP code generation for the database table pages) Manual coding in R and PHP (with source code stored in C:\MicroSatServer)$
NRICARIBOU

nricaribou.cc.umanitoba.ca

Multi-role virtual server

Overview and guide

Paul Galpern

[email protected]

Prepared for: Micheline Manseau

September 7, 2011

NRICARIBOU Overview and Guide (September 7, 2011) 2

Please note:

For security reasons, usernames and passwords in this

document have been removed and replaced with a

reference code. Use the document sent under separate

cover to find these credentials.


An overview of the

structure of the

NRICARIBOU

virtual computer, with

some simplification.

See next pages for

further description.

Boxes indicates an

entry point, tool or

public function

associated with

NRICARIBOU.

The figure also

attempts to show

hierarchical

relationships among

these elements.

NRICARIBOU (a virtual computer/server running Windows Server 2003 operating system) Windows computer name: \\nricaribou DNS name: nricaribou.cc.umanitoba.ca; Aliases: lecol-ck.ca, pademp.com Static IP address: 130.179.18.163

World wide web server Apache software

(d) http://nricaribou.cc.umanitoba.ca/mss/ website containing MicroSatServer web application administered using several tools that can be accessed from the NRICARIBOU desktop:

PHPMaker (template design and automatic PHP code generation for the database table pages) Manual coding in R and PHP (with source code stored in C:\MicroSatServer) Windows Server scheduled tasks that run Windows Powershell scripts

Database server MySQL software contains many databases (e.g. for MicroSatServer, lecol-ck.ca, etc.)

File server FileZilla FTP software Windows VPN software

(a) http://www.lecol-ck.ca website administered using Catalyst web application

(b) http://www.pademp.com website administered using WordPress web application

R programming language A multi-purpose scripting tool that interfaces with PHP and the database server

PHP programming language A multi-purpose scripting tool that interfaces with the world wide web and the database server

University of Manitoba IST Virtual Computing Facility Runs many virtual computers using the VMWare operating system

(c) http://nricaribou.cc.umanitoba.ca/allelematch/ website administered using WordPress web application

1

3

2

5 4

6

7

81 9

100

111

Entr

y p

oin

t

Too

ls

Pu

blic

fu

nct

ion

s


1. University of Manitoba IST Virtual Computing Facility

Purpose: Provides the "cloud computing" or virtual computing framework (i.e. a facility to share

CPU, disk and network resources among multiple "virtual" computers). Entering via this

level of the hierarchy provides the most power. For example can recover complete

backups of the entire NRICARIBOU computer, as well as accessing the NRICARIBOU

desktop. Andrew Perchaluk ([email protected]) at IST can provide assistance at

this level of the hierarchy, including issues with the Windows 2003 Server operating

system.

Control via: VMWare vSphere client: vcenter.cc.umanitoba.ca

Username: <username1>

Password <password1>

2. NRICARIBOU

Purpose: This is a virtual windows computer that runs all the web, database, and other relevant

software. This can be accessed using the vSphere client. Andrew Perchaluk

([email protected]) at IST can provide assistance at this level of the hierarchy,

including issues with the Windows 2003 Server operating system.

Control via: VMWare vSphere client: vcenter.cc.umanitoba.ca OR

Remote desktop protocol client: nricaribou.cc.umanitoba.ca accessed only from a U.

Manitoba IP address. Note same username/password is used for vSphere and for

windows logon.

Username: <username1>


3. World wide web server

Purpose: Uses Apache software to send web pages when the computer receives an http:// request

through the network. Control at this level consists of either stopping/restarting the

Apache web server on the rare occasion it crashes, or when necessary, tweaking the

Apache configuration files. These tools can be found in c:\xampp\apache\.

Control via: NRICARIBOU desktop

4. Database server

Purpose: A database running MySQL software that can store and quickly access large amounts of

data. Data can be queried, inserted, and changed using SQL statements. Data used to run

websites is stored here, as well as data that is stored for scientific purposes

(MicroSatServer data).

Control via: MySQL Query browser: nricaribou.cc.umanitoba.ca on port 3306 OR

Web-based query browser: http://nricaribou.cc.umanitoba.ca/phpmyadmin/

Username: root



5. File server

Purpose: Provides remote access to the files on the NRICARIBOU computer. There are two means

of access: FTP and VPN (virtual private network, which lets the user treat folders on

NRICARIBOU as if they were on their own computer).

Control via: FileZilla FTP client: nricaribou.cc.umanitoba.ca using an active connection OR

Windows VPN client: nricaribou.cc.umanitoba.ca (then go to \\nricaribou in file

explorer) accessible only from a U. Manitoba IP address

Username: <username1> (for both FTP and VPN)

Password <password1> (for both FTP and VPN)

6. PHP programming language

Purpose: A scripting language that is run by the web server, and is used to build web pages in real

time, often taking information from a database. It can also read and write files on

NRICARIBOU. Modifications to PHP code or configuration settings need to be done in a

text editor, and can be accessed from the NRICARIBOU desktop.


7. R programming language

Purpose: A statistical programming language that for convenience was used to code the

MicroSatServer web application. Like PHP it can also access the database directly, and

read and write files on NRICARIBOU. It interfaces with the web via PHP (i.e. it is called

by PHP, and sends its information to the web through PHP). Modifications to R code

need to be done in a text editor, and these can be accessed from the NRICARIBOU

desktop.


8. http://www.lecol-ck.ca

Purpose: Micheline's website. It runs using the Catalyst Content Management system (which in

turn runs using PHP and the MySQL database). Content can be updated via the link given.

Please see the appendix for additional passwords that are required to manage this website.

In particular those that are used to access the Zonedit nameserver which creates the link

between the lecol-ck.ca domain name and the NRICARIBOU IP address.

Control via: http://lecol-ck.ca/admin/

Username: See Micheline Manseau

Password See Micheline Manseau


9. http://www.pademp.com

Purpose: A website created by Sones for the Peace-Athabasca Delta Ecological Monitoring

Program. It uses the WordPress content management system (which in turn uses PHP and

the MySQL database). Content can be updated via the link given. Please see the

appendix for additional passwords that are required to manage this website. In particular

those that are used to access the Zonedit nameserver which creates the link between the

pademp.com domain name and the NRICARIBOU IP address.

Control via: http://pademp.com/wp-admin/

Username: See Micheline Manseau

Password See Micheline Manseau

10. http://nricaribou.cc.umanitoba.ca/allelematch/

Purpose: A website created by Paul Galpern to share his allelematch software for R, and allow

others to download the source code. It uses the WordPress content management system

(which in turn uses PHP and the MySQL database). Content can be updated via the link

given.

Control via: http://nricaribou.cc.umanitoba.ca/allelematch/wp-admin/

Username: admin


11. http://nricaribou.cc.umanitoba.ca/mss/

Purpose: The MicroSatServer web application. It provides a front-end to the genetic data stored in

the database server, as well as analysis tools specifically for this genetic data. Menu and

display pages are coded in PHP. Database table display pages are automatically generated

using the PHP templating tool PHPMaker. All analysis routines are coded in R.

Automated tasks (such as database table synchronization, backups and temporary file

deletion) are controlled using the Windows 2003 Scheduled Task application which in

turn runs Windows Powershell scripts. Modifying the functioning of this web application

is largely a programming task. Generally all tools required to modify the web application

can be accessed from the NRICARBIOU desktop. The operating web application requires

username/password credentials to access.



Technical at a glance

IP address: 130.179.18.163 (static)

DNS names: nricaribou.cc.umanitoba.ca, lecol-ck.ca, ikpodcasts.lecol-ck.ca,

inuitknowledge.lecol-ck.ca, pademp.com

Open TCP ports (all IP addresses): 80 (HTTP), 20 (FTP), 21 (FTP), 3306 (MySQL)

Open TCP ports (restricted to U. Manitoba IP addresses): 3389 (RDP), 1723, 3306 (VPN)

Internet address registry: nricaribou.cc.umanitoba.ca (through umanitoba.ca), *.lecol-ck.ca

(CIRA through Visual Lizard?), pademp.com (?)

Nameservers: nricaribou.cc.umanitoba.ca (umanitoba.ca), *.lecol-ck.ca (zonedit.com1),

pademp.com (zonedit.com1)

Virtual platform: VMware vCenter server v4.1.0

Operating system: Windows Server 2003 R2 Enterprise Service Pack 2

Web hosting platform: XAMPP (Apache 2.0/PHP v5.3.0)

Database platform: MySQL 5.1.37

FTP platform: FileZilla server 0.9.32

Other Windows server roles: Routing and Remote Access (Firewall and VPN)

Backup and recovery: Weekly backup of entire NRICARIBOU hard disk. Most recent backup

is held (VMware). Daily backup of microsatserver, lecol-ck, inuitknowlege, and ikpodcasts

databases. A complete sequence of backups of these databases is retained.

University of Manitoba IST support: Andrew Perchaluk ([email protected])

administers Windows virtual machines, and ensures that the Windows operating system is

regularly updated. Any issues with the server's virtual hardware (i.e. disk, network, CPU),

network security or with the operating system can be addressed to him. Issues with any other

software on NRICARIBOU, including firewalls, the web and database servers should be

addressed to Paul Galpern ([email protected]).

1See Appendix for access to zonedit.com nameservers.

mailto:[email protected]

mailto:[email protected]


How-to guides

How can I access the NRICARIBOU Windows desktop?

How can I move a file onto or off of NRICARIBOU?

How can I move a file onto NRICARIBOU so someone else can download it through the web?

How can I build a mini-website on NRICARIBOU to share files?

How can I restart MicroSatServer because it is not responding?

How can I query the MicroSatServer database?

Show me some SQL queries that I can do on the MicroSatServer database.

How can I recover a database backup for MicroSatServer?

How can I create a username/password to permit someone new to access to the MicroSatServer

web application?

I am a programmer tasked with modifying the MicroSatServer web application. Where do I

start?


How can I access the NRICARIBOU Windows desktop?

There are two ways of doing this. The second (using an RDP client) is more technical, and will

not be explained here. (Note that NRICARIBOU blocks an RDP connection unless it originates

from a U. Manitoba IP address.)

1. Login to a computer on which the VMWare vSphere client is installed (e.g. GAMMA in

Micheline's office). You can install the client on a new computer by downloading the software

from: http://vcenter.cc.umanitoba.ca. The computer should not be on a network with a highly

restrictive firewall.

Windows login information for GAMMA in Micheline's office:

username: Micheline Manseau

password: <password1>

2. Run the vSphere client, and login using the following information:

IP address / Name: vcenter.cc.umanitoba.ca

User name: <username1>

Password: <password1>

3. Click on the "Virtual Machines" tab, and then right-click on nricaribou. Select Open Console.

You may have to click on Hosts and Clusters prior to clicking on Virtual Machines.

http://vcenter.cc.umanitoba.ca/


4. The screen of NRICARIBOU should appear. Likely it will be showing the screen saver.

Click on the screen, and then press CTRL-ALT-INSERT to login. Note that it says to press

CTRL-ALT-DELETE to login, but because you are remote use INSERT instead.

5. You are now logging on to Windows. Use the following credentials:

username: <username1>


6. The desktop of NRICARIBOU should appear. On the desktop are several shortcuts that may

be useful. This virtual computer should be treated like any other computer.

7. Make sure to logoff Windows when you are finished. Also exit the vSphere client for

security reasons.


How can I move a file onto or off of NRICARIBOU?

There are numerous reasons why this might be necessary. There are two ways to do this. FTP is

reliable and easy, but it is also slow and best suited for a small number of files. VPN is the way

to go for frequent file transfers. There is a folder on NRICARIBOU where files are

uploaded/downloaded to and from. See the end of this section.

Moving files using FTP

1. Make sure that the computer from which you are sending or receiving the files has an FTP

client installed, and is not behind a restrictive firewall. FileZilla has been installed on GAMMA

in Micheline's office for convenience. You can install FileZilla to another computer by

downloading the client from: http://filezilla-project.org/download.php. Other FTP clients will

also work.




2. Load FileZilla. Enter the following information, and press Quickconnect.

host: nricaribou.cc.umanitoba.ca


passowrd: <password1>

3. If this is a new installation you will also have to set it to Active mode. Go to

EditSettingsFTPActive.

4. Drag the files to the yellow area on FileZilla and they will be uploaded to NRICARIBOU.

http://filezilla-project.org/download.php


5. To download files from FileZilla, click on a file in the yellow area and drag it to the desktop

or some other location on your computer.

Moving files using a VPN (Virtual Private Network)

Steps for doing this from GAMMA in Micheline's office are given, as VPN functionality has

already been established on this machine. To install VPN on another computer requires many

manual steps, including tweaking firewalls. Instructions are not given.

1. Login:




2. Click on "nricaribou_vpn" icon on GAMMA. Then click on "manseau_transfer_ftp on

nricaribou" icon on GAMMA.

3. The resulting window that opens is like any windows file

folder. Files can be copied to or from this folder.

Where do the files I transfer go to/come from on NRICARIBOU?

All files transferred using FTP or VPN illustrated above are coming from, or going to a folder on

the desktop of NRICARIBOU called manseau_transfer_ftp.

To access this folder on NRICARIBOU:

1. Login to the NRICARIBOU desktop (see: How can I access the NRICARIBOU Windows

desktop?)

2. Click on the manseau_transfer_ftp folder on the desktop.

3. Move files to or from this folder in the usual manner.


How can I move a file onto NRICARIBOU so someone else can

download it through the web?

This is very useful when a file must be sent that is too large for email. Or, when you wish to

make some materials available for others to download at their convenience.

1. Follow steps from How can I move a file onto or off of NRICARIBOU?.

2. Access the NRICARIBOU desktop (see: How can I access the NRICARIBOU Windows

desktop?) File will now be stored in the folder manseau_transfer_ftp visible on the desktop.

3. Move the file to one of two locations depending on how you want others to access it.

a) To have users download a file at nricaribou.cc.umanitoba.ca move the file you just

transferred to the folder c:\xampp\htdocs. You can tell others to download this file

(abcdefgh.zip) using their browser. The address for this file would then be:

http://nricaribou.cc.umanitoba.ca/abcdefgh.zip

b) To have users download a file at lecol-ck.ca move the file you just transferred to the

folder c:\xampp\htdocs\lecol-ck. You can tell others to download this file (abcdefgh.zip)

using their browser. The address for this file would then be:

http://lecol-ck.ca/abcdefgh.zip

Note that, when giving a web address for your file, the capitalization must be exact. For

simplicity, don't create files for web download that have spaces or unusual characters in

their names. It is also good practice to go with all lower case in order to avoid typing

errors.


How can I build a mini-website on NRICARIBOU to share files?

There is a conference coming up and you want to share some files. Or a course, and we need a

repository for papers and other reference material. A simple page with links to these files

constitutes a website and can quickly be put together by moving the files onto NRICARIBOU,

and creating an index page in HTML (Microsoft Word can be used to save pages in this format).

1. Create an index page using Microsoft Word. You can use this file as a template and open it in

Microsoft Word. The index can be very simple: some text, with some of the words hyperlinked.

Hyperlink by highlighting a word or words and pressing CTRL-K in Microsoft Word, and

entering the filename and extension you want to link in the "Address" part of the dialog (no path

is required).

2. Save the file as web page in Microsoft Word with the name index.html. This will create a file

with this name and a folder called index_files in the folder where you saved it.

3. Move this file (index.html) and the folder it created (index_files) and all other files that are

hyperlinked on it onto NRICARIBOU. Follow steps from How can I move a file onto or off of

NRICARIBOU? This process will also work for folders containing files.


desktop?) Files will now be stored in the folder manseau_transfer_ftp visible on the desktop.

5. Go to the folder c:\xampp\htdocs\ and create a new folder with a short name that will be the

de facto name of your website. New folder example: c:\xampp\htdocs\examplesite\.

6. Copy the files from manseau_transfer_ftp to the new folder.

7. Using your browser go to http://nricaribou.cc.umanitoba.ca/whatever/, replacing whatever

with the name of the new folder. The browser should automatically load the index page you

created, and the links should load the files.

Note that when naming files and folders for the web capitalization must be exact. Don't create

files for web download that have spaces or unusual characters in their names. It is also good

practice to go with all lower case in order to avoid typing errors.

8. You can advertise this link as your mini-website!

http://nricaribou.cc.umanitoba.ca/examplesite/index.html


How can I restart MicroSatServer because it is not responding?

You will know the MicroSatServer web application is non-responsive if you try to login in the

usual way (e.g http://nricaribou.cc.umanitoba.ca/mss/) and after you enter your information

nothing happens for more than 5 seconds. This occurs because the Apache web server software

has crashed. The reason for the crash is complex, but ultimately it is caused by a bug in the

Apache software. Many attempts have been made to work around this bug with good success,

but likely such freezes will still appear from time to time.

The following steps are required to restart the Apache web server software. This will also restart

MicroSatServer.

1. Login to the NRICARIBOU desktop (see: How can I access the NRICARIBOU Windows

desktop?)

2. Click on the "Control web and ftp server" icon.

3. The following dialog should appear. If you receive an error message, then it is already active.

So, click on the orange icon near the clock.

4. Click the Stop button for Apache (highlighted above).

A message saying the server has stopped should appear

below. Wait about 30 seconds. Click the start button for

Apache. In about 5 seconds the word Running should

appear highlighted in green. If it does not appear, repeat

these steps. Patience is key. Eventually the server will

restart and "Running" will appear indicating everything is

okay.

5. Go to http://nricaribou.cc.umanitoba.ca/mss/ on any

computer and check you can login normally.

6. If nothing seems to work, do an old-fashioned reboot. Click on the Start button on the

NRICARIBOU desktop. Then click the shut-down button. Make sure you choose restart in the

drop-down box. You'll need to enter a few random characters under "Comments" When the

computer restarts all the servers will start too, and you should be able to login again.

http://nricaribou.cc.umanitoba.ca/mss/

http://nricaribou.cc.umanitoba.ca/mss/


How can I query the MicroSatServer genetic database?

Background

There are two types of MySQL databases on NRICARIBOU: (1) website content databases;

and, (2) scientific databases. The first type can generally be ignored because content

management software maintains and queries them. MicroSatServer is a web application built

around a database of the second type, and occasionally queries will be required that are not

performed automatically by MicroSatServer.

Two methods for sending SQL queries to NRICARIBOU databases are given. The first uses a

web application called phpMyAdmin that will work from locations with highly restrictive

firewalls. A second alternative uses MySQL client software. It is fast, efficient, and

recommended for serious users, but will not work from restricted locations.

Using the phpMyAdmin web application

1. Open a web browser on any computer, and go to:

http://nricaribou.cc.umanitoba.ca/phpmyadmin/

2. Use the following credentials to login:

username: root


3. Click on microsatserver in the left panel to

browse and query the MicroSatServer database.

4. Click on a table in the left panel to begin

browsing the contents of that table.

5. Select the SQL tab to enter an SQL query. For

example enter the following and click "Go"

SELECT * FROM pellet WHERE population_id = 'North Interlake' AND collection_year =

2004;

6. Many additional operations on database tables can be performed using the tabs. Note

specifically Import and Export which can interface with Excel using CSV files.

http://nricaribou.cc.umanitoba.ca/phpmyadmin/


Using the MySQL client web application

This has been installed on GAMMA in Micheline's office. If you wish to install this software on

a Windows computer, please download an installation package from this address:

http://nricaribou.cc.umanitoba.ca/repository/ mysql-gui-tools-5.0-r17-win32.msi. Other MySQL

clients are also available online and work in a similar manner.

1. Open the MySQL Query browser.

2. Enter the following:

host: nricaribou.cc.umanitoba.ca

port: 3306

username: root


default schema: microsatserver

3. Queries can be entered in the top left box, and results appear in the bottom box.

4. It is also possible to edit a datum in a table by clicking on Edit at the bottom of the screen,

making changes as if using a spreadsheet, and then clicking Apply changes. Keep in mind it is

not possible to "undo" a change to a database table as it would be using spreadsheet software.


Show me some SQL queries that I can do on the MicroSatServer genetic

database

The MicroSatServer web application automatically handles most of the common queries on the

database. On occasion, however, it may be necessary to find records in a table that fulfil certain

complex criteria, or to use the contents of one table to look up information on a second table.

Another common query is to replace the fields in multiple records with a new value, avoiding

tedious manual editing of each record in MicroSatServer.

Structured Query Language (SQL) qualifies as a simple programming language, although

programming experience is not required to use it. An SQL query is most often a command

written in English that asks the database to find records (SELECT), add records (INSERT),

remove records (DELETE), or change records (UPDATE) in a table. The format of these

commands is highly structured, and many excellent references and tutorials are available on the

web that explain this syntax. A good place to start is the MySQL reference manual:

http://dev.mysql.com/doc/refman/5.1/en/.

To perform any of the following example queries first open phpMyAdmin or a query browser by

following the steps in How can I query the MicroSatServer database? The intention, here, is not

to give an introductory tutorial in SQL, but rather indicate some common applications of queries

and demonstrate broadly what is possible.

Example 1: View all records in a table

SQL

Query

SELECT * FROM goldgenotype

SELECT means that records should be returned

* means return all columns (or fields) from the table

goldgenotype indicates the name of the table

The full analysis-ready "gold" genotype table should be returned (where -99 indicates data that

is not "gold" or missing due to amplification failure)

http://dev.mysql.com/doc/refman/5.1/en/


Example 2: View specific records from a table

SQL

Query

SELECT * FROM pellet WHERE (population_id = 'Brazeau' OR population_id =

'Maligne' OR population_id = 'Tonquin') AND collection_year IN (2004, 2005,

2006, 2007, 2008, 2009, 2010)


* means return all columns (or fields) from the table

pellet indicates the name of the table

WHERE indicates that a series of conditions follow

Conditions are in the form of the column name, and operator (=, <, > etc.) and a value. If the

value is text it should be surround in single quotation marks. To specify a range of conditions

IN followed by comma separated list surrounded in brackets is given.

AND and OR are logical terms

Only the specific records from the pellet collection table are returned

Example 3: View specific columns from a table

SQL

Query

SELECT pcid, gold_flag FROM bm848 WHERE (bm848a=-99 OR bm848b=-99) AND

batch_id = 'Tonquin 2009'


pcid, gold_flag means return only these columns

bm848 indicates the name of the table


This query returns the PCID and gold flag status for all missing (-99) bm848 genotypes in the

Tonquin 2009 batch.

Example 4: View specific records from a table using a fragment of text

SQL

Query

SELECT pcid, abi_filename FROM map2c WHERE abi_filename REGEXP 'mkabi'


pcid, abi_filename means return only these columns

map2c indicates the name of the table


REGEXP indicates that the condition will be met if the text fragment mkabi is found in the

abi_filename column of a record

This query returns the PCID and abi_filenames in the map2c table for data generated by Marina

Kerr (mkabi)


Example 5: View specific records from a table and remove duplicates

SQL

Query

SELECT DISTINCT rt5a FROM rt5 WHERE batch_id REGEXP 'North Interlake' ORDER

BY rt5a ASC


DISTINCT means throw out exact duplicates

rt5a means return only this column for the selected records

rt5 indicates the name of the table


ORDER BY indicates that the results should be sorted by the given column

ASC indicates that this sort should be in ascending order

This query returns the unique alleles in rt5 for the North Interlake population in increasing order

Example 6: Count how many records meet a simple criterion

SQL

Query

SELECT population_id, collection_year, count(PCID) FROM pellet GROUP BY

population_id, collection_year


population_id, collection_year means return only these columns

count(PCID) indicates that a count should be returned

GROUP BY indicates records should first be grouped according to two criteria before counting the

number in each group

This query returns the number of pellets collected by population and collection year

Example 7: View records from one table using criteria from another table

SQL

Query

SELECT pcid, utm_easting, utm_northing FROM pellet WHERE PCID IN (SELECT PCID

FROM rt5 WHERE rt5a=100 OR rt5b=100)


pcid, utm_easting, utm_northing means return only these columns

pellet indicates the name of the table


(SELECT ...) indicates a sub-query that returns only PCID which is then used as a criterion for

the first query.

This query returns the sampling locations of individuals with a very rare rt5 allele (100).


Example 8: Count how many records meet a complex criterion

SQL

Query

SELECT population_id, collection_year, count(PCID) FROM goldgenotype WHERE

PCID IN (SELECT PCID FROM pellet WHERE sampled_by regexp 'Kent') GROUP BY

population_id, collection_year;


population_id, collection_year, means return only these columns

count(PCID) indicates that a count of the index should be returned

goldgenotype indicates the name of the table (the analysis-ready genotype table)


(SELECT ...) indicates a sub-query that returns only PCID which is then used as a criterion for

the first query.

GROUP BY indicates records should first be grouped according to two criteria before counting the

number in each group

This query returns the number of partial or complete genotypes in the database by population

and collection year in which someone name Kent was credited as doing the pellet sampling. In

other words: how much credit to we owe Kent for building our analysis-ready dataset!

Example 9: Create a new table

SQL

Query

CREATE TABLE example (id INT, firstname VARCHAR(50), lastname VARCHAR(50),

score FLOAT)

CREATE TABLE means make a new table

example indicates the name of the new table

The remaining information in brackets give the name of the columns followed by the type of

column separated by commas. Please see MySQL documentation for details. An alternative

means of creating tables by point-and-click is provided by phpMyAdmin.

This creates a table that is used in the subsequent examples.

Example 10: Insert records into a table

SQL

Query

INSERT INTO example (id, firstname, lastname, score) VALUES (1, 'Paul',

'Galpern', 0.95);

INSERT INTO example (id, firstname, lastname, score) VALUES (2, 'Micheline',

'Manseau', 0.96);

INSERT INTO example (id, firstname, lastname, score) VALUES (3, 'New',

'Student', 0);

INSERT INTO means add a record to an existing table


The first set of brackets gives the name of the columns separated by commas.

VALUES indicates that values for those columns follow in brackets

Semi colons at the end of each line indicate that multiple SQL statements are given together

An alternative means of inserting records by point-and-click or by CSV import is provided by

phpMyAdmin and the MySQL client.

This adds three records to the table example


Example 11: Change records in one or multiple records in a table

SQL

Query

UPDATE example SET score = 0.95 WHERE firstname = 'New' and lastname =

'Student'

UPDATE means change a record or records in an existing table


SET indicates that a column name and value follow

WHERE gives the conditions under which the value in that column should be changed

This changes the score for only one record, as there is only one that meets this condition

An alternative means of editing single records by point-and-click is provided by phpMyAdmin

and by the MySQL client.

NOTE THAT WHEN A FIELD IS CHANGED NO BACKUP IS KEPT OF THE

PREVIOUS VALUE. UPDATE SHOULD BE DONE WITH CAUTION. FIXING AN

INCORRECT UPDATE REQUIRES RECOVERING A DATABASE BACKUP

Example 12: Delete records in a table

SQL

Query

DELETE FROM example WHERE firstname = 'New' and lastname = 'Student'

DELETE FROM means delete a record


WHERE gives the conditions under which the record should be deleted

An alternative means of deleting records by point-and-click is provided by phpMyAdmin and by

the MySQL client.

NOTE THAT WHEN A RECORD IS DELETED NO BACKUP IS KEPT. DELETE FROM

SHOULD BE DONE WITH CAUTION. REVERSING THIS REQUIRES

RECOVERING A DATABASE BACKUP

Example 13: Delete an entire table

SQL

Query

DROP TABLE example

NOTE THAT WHEN A TABLE DROP IS USED NO BACKUP IS KEPT. DROP SHOULD

BE DONE WITH CAUTION. REVERSING THIS REQUIRES RECOVERING A

DATABASE BACKUP


How can I recover a database backup for MicroSatServer?

Disaster can strike the MicroSatServer database in many--fortunately uncommon--ways. The

most likely case is a manual SQL query that has been improperly formed. Equally, incorrect

score data could be uploaded by a user, and determining which records this represents can

sometimes be extremely tedious. Because the database does not remember its previous state, it is

not possible to quickly "undo" the last action as it might be in a word processor or spreadsheet

program. In a variety of scenarios it will be desirable to revert to the last backed-up state of the

database.

Fortunately, the entire contents of the MicroSatServer database are backed-up daily at midnight,

and a record of every backup is retained, making it possible to revert to the state of the database

as it was at any point in the past. Keep in mind that while reverting to a backup made

immediately before the event will fix the problem, it will also remove any desired changes

to the database that have been made in the interim. Use with caution!


desktop?)

2. Find the folder: D:\MicroSatServer\Backups\MySQL\Backups

3. Find the backup you wish to restore to (the most recent will be this morning). They are

named as follows: mssYYMMDD_XXXXXX.sql where "mss" indicates MicroSatServer, YY,

MM and DD are two digit representations of the year, month and day respectively, and

XXXXXX is the size of the database in rows.

4. Copy this backup file to the folder: C:\xampp\mysql\bin

5. Go to the start button and click on Run.

6. Type "cmd" (without quotes)

which runs the shell (right).

7. Type: "cd\xampp\mysql\bin"

8. Type: "mysql –u root –p microsatserver < mssYYMMDD_XXXXXX.sql" replacing the capital

letters as appropriate. Then enter the password: <password2>

10. The database will be restored to its state on YYMMDD at 12:00am.


How can I create a username/password to permit someone new

to access the MicroSatServer web application?

The MicroSatServer web application requires a username and password to access it. Granting a

new user access must be done using an SQL query. Two levels of user access are possible.

1. Open phpMyAdmin or a query browser by following the steps in How can I query the

MicroSatServer database?

1. View the existing usernames and passwords. The passwords appear in an encrypted form.

SELECT * FROM caribou_genetics_users

2. Enter the following SQL statement, replacing newuser with a unique username for the person

(typically first initial last name is used). Note that it is necessary to type the long string of

characters giving the encrypted password exactly as shown.

INSERT INTO caribou_genetics_users (username, password, userlevel) VALUES ('newuser',

'4c7e0210b2cc9120dd2a8989677ff205', -1)

3. The new user should be able to login using the username you gave them and the default

password: ran91fer. They should promptly change this password to something else using the

link on the left panel in the MicroSatServer web application.

4. It is also possible to grant access to someone who can only view the analysis-ready results.

Replace -1 with 3 in the query above. Userlevel management has not been robustly

implemented, however, so a malicious and determined person with even this reduced access

could find a way to change some elements of the database.


I am a programmer tasked with modifying the MicroSatServer

web application. Where do I start?

The goal of this "Guide" document was to describe the NRICARIBOU computer, and not to

focus on the MicroSatServer web application. Indeed, this is another document in itself! In the

event that such a document does not get written, the following should provide some basic

information from which a programmer could build an understanding of how the web application

operates. Comments included in the PHP and R code may also be helpful.

Below the most important files associated with running the web application are listed, with some

brief comments on their purpose and how they interact with others. Apologies that is not

comprehensive or detailed, but hopefully it provides a place to start!

C:\xampp\htocs\mss\ This is the root folder of the web application and contains

the public interface of the web application

*list.php

*info.php

*delete.php

These files are generated by the PHP templating tool

PHPMaker v6. This version of the software must be used

to edit them. The master template file is C:\Documents

and Settings\pgalpern\My

Documents\PHPMaker\Projects\MicroSatServer_v1.pmp

No changes to these PHP files should be attempted

manually. Use PHPMaker exclusively for this purpose.

Numerous defaults have been tweaked, so the installed

version on the server should be used. In the case of the

pages named for genotype loci, PHPMaker produces a

series of locus*.php templates, and routines in R are

manually called to roll out the PHP pages and search and

replace "locus" as required (see microsatserver_v1.r)

menu.php

This is the menu page of the MicroSatServer application.

It and its associated includes are the nexus of

MicroSatServer. Using a database lookup it "includes"

any of a number of pages.

mssToR_public.php

This is the configuration file for MicroSatServer's PHP

interface with R. It establishes how PHP will call R

routines. (i.e. using PHP system() calls, and sending

POST data to a MySQL database intermediary)

C:\microsatserver\PHP This folder contains the PHP interface for

MicroSatServer analyses and additional functions coded

by Paul Galpern ([email protected]) .


mssToR.php This is the PHP side of the MicroSatServer to R interface

db_fsa.php Routines to associated with the upload and download of

FSA files from the FSA repository.

db_fsa.php Routines to associated with the upload and download of

FSA files from the FSA repository.

importgenetic.php PHP interface to R for importing genotype data

importpellet.php Routines to import pellet data (100% PHP)

importtelem.php Routines to import telemetry data (100% PHP)

mss_analysis.php PHP interface to R for analyses contained in the

"Analysis-ready" section of MicroSatserver

mss_automation.php PHP interface to R synchronization of database tables.

This is called twice a day by Windows Task Scheduler,

and also called when a manual synchronization is

requested.

mss_sandbox.php PHP interface to R Technician's Sandbox routines

C:\microsatserver\R This folder contains the R routines for MicroSatServer

analyses and additional functions coded by Paul Galpern

([email protected]) .

mssToR.R This is the R side of the MicroSatServer PHP to R

interface

microsatserver_v1.R The guts of MicroSatServer. Contains R code that

provides most of the additional functionality and analysis

routines.

pg_genetic.R Some genetic support routines called by

microsatserver_v1.R

C:\microsatserver\PS1 This folder contains some of the Windows Powershell

routines that are called by the Windows Task Scheduler

mss_tmpfilecleanup.ps1 Called every hour by the Windows Task Scheduler to

delete temporary files that are older than 24 hours.

D:\microsatserver Folder on D drive containing file repository and backups

for MicroSatServer. Stored on this drive as there is more

storage space.

D:\microsatserver\backups\mys

ql

Root folder for the daily backups of MicroSatServer

microsatserver_backup.ps1 Windows Powershell script called at midnight by the

Windows Task Scheduler to create SQL backups of

MicroSatServer

D:\microsatserver\abifiles Folder in which all the ABI raw data is stored as ZIP

files. This folder is controlled directly by the File

Repository section of the MicroSatServer application.


Appendix 1 – Additional information for *.lecol-ck.ca

1. Domain name registration for lecol-ck.ca

username/password (?)

This registration will be held by CIRA, and likely managed by Visual

Lizard until a transfer in management is initiated and approved.

2. FTP Account

username: lecol


Full privileges for file and folder creation/deletion etc.

3. MySQL Account

username: lecol


database: lecol, inuitknowledge, ikpodcasts

Full privileges for only these three databases

Can be accessed at localhost using

http://nricaribou.cc.umanitoba.ca/phpmyadmin or remotely using a MySQL

client

4. DNS Service

Zonedit.com (http://zonedit.com)



domains: lecol-ck.ca, www.lecol-ck.ca, inuitknowledge.lecol-ck.ca,

ikpodcasts.lecol-ck.ca

nameservers: ns13.zoneedit.com (66.223.40.121), ns9.zoneedit.com

(66.240.231.42)

These nameservers redirect requests for the above domains to

nricaribou.cc.umanitoba.ca (130.179.18.163)

5. Virtual hosting at nricaribou.cc.umanitoba.ca

Modifications to virtual hosting require manual editing of the

c:\xampp\apache\conf\extra\httpd-vhosts.conf on nricaribou.

Web server at nricaribou.cc.umanitoba.ca will serve pages from the

following folders, according to the domains from which the page is

requested:

lecol-ck.ca,

www.lecol-ck.ca (http://nricaribou.cc.umanitoba.ca/lecol-ck/)

inuitknowledge.lecol-ck.ca (http://nricaribou.cc.umanitoba.ca/lecol-

ck/inuitknowledge/)

ikpodcasts.lecol-ck.ca (http://nricaribou.cc.umanitoba.ca/lecol-

ck/ikpodcasts/)