f5228 beowulf cluster server - log book report

UNGKU OMAR PREMIER POLYTECHNIC

DEPARTMENT OF POLYTECHNIC EDUCATION

MALAYSIAN MINISTRY OF HIGHER EDUCATION

FINAL PROJECT LOG BOOKDIPLOMA IN INFORMATION TECHNOLOGY (NETWORKING)

INFORMATION AND COMMUNICATIONS TECHNOLOGY DEPARTMENT

DEPARTMENT OF POLYTECHNIC EDUCATION

Muhamad Asyraf Bin Ramli 01DNS09F2043Muhammad Farid Bin Othman 01DNS09F2081

TABLE OF CONTENTS :

CHAPTERS CONTENTS PAGE(S)

1.0 Table of Contents 2

2.02.12.2

Student InformationMuhamad Asyraf Bin RamliMuhammad Farid Bin Othman

34

3.0 Introduction 5

4.0 Student Reminders 5

5.0 Meeting Duration 5

6.06.16.26.36.46.56.66.76.86.96.106.116.126.136.146.156.16

Weekly ReportsWeek 1Week 2Week 3Week 4Week 5Week 6Week 7Week 8Week 9Week 10Week 11Week 12Week 13Week 14Week 15Week 16

6 - 21

6.0 Conclusion 22


INFORMATION TECHNOLOGY AND COMMUNICATION DEPARTMENTUNGKU OMAR PREMIER POLYTECHNIC

MALAYSIAN MINISTRY OF HIGHER EDUCATION

STUDENT INFORMATION :

YEAR / SESSION June 2012

STUDENT INFORMATION

NAME Muhamad Asyraf Bin Ramli

REGISTRATION NUMBER

01DNS09F2043

COURSE Diploma in Information Technology (Networking)

ADDRESS No. 9, Persiaran Wira Jaya Barat 16, Taman Ampang, 31400 Ipoh, Perak Darul Ridzuan

CLASS DNS6B

PHONE NUMBER 017-524 9834

PROJECT INFORMATION

TITLE Beowulf Cluster Server

SUPERVISOR Mr. Mohd. Hazri Bin Hasnan

Student Signature,

………………………………………….(MUHAMAD ASYRAF BIN RAMLI)

INFORMATION TECHNOLOGY AND COMMUNICATION DEPARTMENT


UNGKU OMAR PREMIER POLYTECHNICMALAYSIAN MINISTRY OF HIGHER EDUCATION

STUDENT INFORMATION :

YEAR / SESSION June 2012

STUDENT INFORMATION

NAME Muhammad Farid Bin Othman

REGISTRATION NUMBER

01DNS09F2081

COURSE Diploma in Information Technology (Networking)

ADDRESS No. 9, Persiaran Wira Jaya Barat 16, Taman Ampang, 31400 Ipoh, Perak Darul Ridzuan

CLASS DNS6B

PHONE NUMBER 012-234 1088

PROJECT INFORMATION

TITLE Beowulf Cluster Server

SUPERVISOR Mr. Mohd. Hazri Bin Hasnan

Student Signature,

………………………………….………….(MUHAMMAD FARID BIN OTHMAN)

INTRODUCTION :


The final project log book is used specifically for writing notes on all the activities and work done by students

throughout the duration the duration of the project. The log book is one of the ways in which the department can

identify and access the attitude, initiative and achievement of the student’s project. It is one of a requirement for

students to complete during the project is carried out either work individually or in a team under supervision of

respective supervisors. Therefore, it is important for students to fully utilize the log book and submit to a panel

of examiners for evaluation.

STUDENTS REMINDERS :

1. The log book must be handed in to the supervisor to be graded and verified at least once in every

week.

2. Students must bring along their log book at every meeting and discussion with their supervisor so that

verification can be done after the session.

3. The log book must be handed in to the supervisor with the final project report for evaluation and

grading.

MEETING DURATION :

The student’s attendance will be taken during SLT by the supervisors as follows:

1. FP521 - 2 hour’s theory and 2 hour’s practical in a week.

2. FP621 – 1 hour’s theory and 3 hour’s practical in a week.


WEEK : 1 DATE : 09/07/2012 – 13/07/2012 TIME : 10:00 A.M.

This report has to be written in every week and enclosed detail attachment to support your report if necessary.

WEEKLY PLANNING :

1) Inquire Mr. Hazri regarding the F5228 project module, expected deadlines for the project and documentation.

2) Conduct research on the possibility of implementing Clustered Servers in networks.

3) Create a working schedule for the development of this project.

WEEKLY ACHIEVEMENT :

1) Incomplete. Project meeting is cancelled and suspended until next week.

2) Completed. Found a lot of useful notes on the web.

3) Completed. Schedule is done in the form of a Gantt chart.

REFLECTIVE :

1) Unable to meet with project supervisor. Will seek Mr. Hazri again next week.

2) We discovered that it is possible to implement Clustered Servers in Local networks and also configuring them to function as a Web Host Servers.

3) The Gantt chart will function as a planned schedule for us to follow while developing our project throughout the course of this semester.

SUPERVISOR’S COMMENT :

1) OK. Do some self-studying for the time being and produce a list of equipments to purchase.

VERIFIED BY :Supervisor’s Signature : Date : 13/07/2012

……………………………..(Mohd. Hazri Bin Hasnan)


WEEK : 2 DATE : 16/07/2012 – 20/07/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Discuss with project supervisor about possible market opportunities for this project.

2) Create a list of hardware and software required for our project.

3) Consult with project supervisor regarding our proposed budget and presenting our list of planned equipment purchases.


1) Completed. Project supervisor advised to focus on SOHOs and small businesses.

2) Completed.

3) Completed. Project supervisor gave the green light to proceed with purchase.

REFLECTIVE :

1) Mr. Hazri had suggested for our project to focus on the scope of small businesses and personal users, and to avoid large businesses and organizations due to restrictions in terms of implementation.

2) List of purchases for hardwares includes 3Desktop PCs, a Switch, UTP cabling and also Linux Operating Systems.

3) Project supervisor agreed with our proposed budget, which totaled just under RM1,700.00 and informed us that it was also possible to procure the equipment at a much cheaper budget.


1) OK, please proceed on to procuring the equipment as listed.




WEEK : 3 DATE : 23/07/2012 – 27/07/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Purchase an Intel i3 Desktop PC to be used as the Master node.

2) Survey market for 2 used Intel Pentium 4 PC units to be used as Slave nodes.

3) Purchase a 5-port Desktop Switch for connection.

4) Purchase 3 pieces of Category 5E Straight-through UTP cables.


1) Completed. Purchased a unit for RM1,100.00.

2) Incomplete. Managed to obtain 1 unit only from a family member for free.

3) Incomplete. Could not find a 5-port Desktop Switch.

4) Complete. Purchased 3 units for RM15.00

REFLECTIVE :

1) Only 1 unit of an Intel Pentium 4 PC was able to be obtained. Will try to look for another unit next week at the Low Yat Plaza in Kuala Lumpur.

2) Unable to procure a 5-port Desktop Switch. Only 8-port and 24-port variants were available at the time. Will try to look again next week at the Low Yat Plaza in Kuala Lumpur.


1) OK. Continue surveying the local IT stores for any other equipment that are still not purchased yet.




WEEK : 4 DATE : 30/07/2012 – 3/08/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Head down to the Lowyat Plaza in Kuala Lumpur to look for the required equipment.

2) Look for a used Intel Pentium 4 PC to be used as a Slave node.

3) Purchase a 5-port Desktop Switch for connection.

4) Obtain a copy of the Linux Ubuntu Operating System.


1) Completed.

2) Incomplete. None found.

3) Completed. Bought a unit from an IT store at the Low Yat Plaza.

4) Completed. Obtained a distributable copy of Linux Ubuntu Ver. 12.0.4 Operating System from a friend.

REFLECTIVE :

1) Unable to find a Desktop PC matching our required specifications and budget. Will try to look for another unit next week in Ipoh.

2) Managed to obtain every hardware equipment except for a unit of a Desktop PC. In the meantime, we will be configuring the 2 Desktop PCs units currently available in our possession.


1) OK. Continue surveying local IT stores and also online trading websites for affordable, secondhand Desktop PCs.




WEEK : 5 DATE : 06/08/2012 – 10/08/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Purchase a used Intel Pentium 4 PC to be used as a Slave node.

2) Install the 2 PC units with the Linux Ubuntu Operating System.

3) Explore the Linux Ubuntu Operating System.

4) Master the basic system configurations on the Linux Ubuntu Operating System.


1) Incomplete. Unable to procure a used Desktop PC.

2) Completed.

3) Completed.

4) Most basic configurations are able to be learned, but still unable to grasp network-related configurations.

REFLECTIVE :

1) Unable to procure a used PC matching our desired specifications. Will continue again next week.

2) Did a lot of self-exploring on the Linux Ubuntu Operating System. Able to understand its method of operation and the hierarchical file structure due to our exposure to the Linux OS environment learnt in the previous semester.

3) Our research on Linux configurations came from a variety of Linux Developer Forums, Blogs and Websites. Currently studying methods on how to configuring the Cluster Server to function as a Web Host server.


1) Good. Try to also consult the ICT department lecturers and staffs for tips and information on setting up a Linux-based server.




WEEK : 6 DATE : 13/08/2012 – 17/08/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Purchase a used Intel Pentium 4 PC to be used as a Slave node.

2) Test connectivity and ensure every hardware is functioning properly.

3) Take screenshots of the Linux OS installation process for use in the User Guide section of the documentation.


1) Completed. Bought a unit for RM260.00 in Ampang, Ipoh.

2) Completed. Successfully able to connect all 3 Desktop PCs and Switch. Also able to ping every device on the Local network

3) Completed. Documented installation procedures.

REFLECTIVE :

1) Finally managed to purchase the last required hardware device, a used Intel Pentium 4 Desktop PC. Installed also with a copy of Linux Ubuntu Ver. 12.0.4. PC is in good working order.

2) Success in connecting the devices and performing Ping tests on each of them shows that the hardwares are working and functioning as intended.

3) The screenshots taken are to be used in the Linux Ubuntu Operating System installation section of the User Guide, in the project documentation.


1) OK. Try to perform documentation of the project while progressing to save time.




WEEK : 7 DATE : 20/08/2012 – 24/08/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Meet up with an IT officer from the Pejabat Tanah Perak in Ipoh to obtain his views regarding the utilization of Cluster Servers in general.

2) Configure NFS and SSH protocols on each of the Linux Ubuntu OS.

3) Experiment with installing a second Network Interface Card on the Master Node.


1) Completed. Received useful insight and understanding of Clustered Server.

2) Incomplete. Unable to configure NFS and SSH because the text editor is unable to read keyboard mappings. Searching for troubleshooting methods.

3) Completed. Installation is straightforward due to the Plug and Play nature of the hardware and compatibility with the Linux Ubuntu OS.

REFLECTIVE :

1) The IT officer shared his experiences with handling Clustered Servers on a large scale, and advised us to research more on Beowulf, and other Parallel Computing software such as MPI and DistCC to help us gain a better understanding of Server Clustering methods.

2) The text editor used in the Linux Ubuntu Ver. 12.0.4 was different from previous versions of Linux, making it difficult for us to properly configure NFS and SSH protocols on the OS.

3) Installing a second NIC allows the Master node to connect to outside networks simultaneously while also connected to the Local clustered network.


1) OK. Search for troubleshooting methods regarding the text editor. Look up on Linux Developer Websites and Forum threads.




WEEK : 8 DATE : 27/08/2012 – 31/08/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Troubleshoot the inability to configure NFS and SSH in the text editor.

2) Meet with ICT lecturer Mr. Mohd. Assidiq Bin Che Ahmad regarding issues in configuring NFS and SSH protocols.

3) Implement solution in all 3 Linux Desktop PCs.


1) Incomplete. Will try to consult an ICT lecturer who specializes in Server Management.

2) Completed. Mr. Assidiq advised us to download and update patches and fixes for our Linux distributions.

3) Completed. Once finished updating patches, configurations could be made.

REFLECTIVE :

1) Our difficulties in configuring NFS and SSH for the OS was due to missing patches and hotfixes in our copy of the Linux Ubuntu Ver. 12.0.4. The completed Linux Ubuntu Ver. 12.0.4 consisted of 4 DVDs, while our copy was only the size of a single DVD.

2) Once finished updating and installing the missing patches, we were able to modify the contents of the text editor. All that needed to be done now was to change the access of the directories using the chmod command.


1) Good. Proceed with current progress and attempt similar troubleshooting methods for other issues in the future.




WEEK : 9 DATE : 03/09/2012 – 07/09/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Generate public and private SSH keys.

2) Ensure that only the Master node’s public key is copied, and not the Slave node’s.

3) Design a suitable topology to be used to arrange our hardware setup.


1) Completed. Successfully broadcasted SSH keys to other nodes.

2) Completed. The public key in use is only the one belonging to the Master node.

3) Completed. Topology can be referred to in the documentation.

REFLECTIVE :

1) The whole week was devoted to understanding the concept and importance of the public and private SSH keys used. These keys are important in order to establish connectivity between the Master node and the two other Slave nodes, in this case, only the public key belonging to the Master key is used.

2) The topology diagram was designed using Microsoft Visio and also supplemented by Adobe Photoshop for touching up.


1) OK. Continue with the next phase of the project development.




WEEK : 10 DATE : 10/09/2012 – 14/09/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Apply patches to every Linux Ubuntu Ver. 12.0.4 Operating Systems in each node.

2) Search for a solution to fix the current problem involving the OS.

3) Repair or reinstall the Linux Ubuntu Ver. 12.0.4 Operating Systems.


1) Incomplete. Unable to apply patches

2) Incomplete. Unable to find a suitable solution.

3) Complete. Successfully reformatted and reinstalled the Operating Systems.

REFLECTIVE :

1) While trying to apply a patch to each of our Linux Ubuntu Ver. 12.0.4 Operating Systems, an error occurred and the OS is unable to boot properly. Solution is to try and troubleshoot the problem.

2) We were unable to provide a suitable solution to our problems regarding the corrupted Operating System, and therefore we had no choice other than to choose to reformat the partitions and install a fresh copy of the Linux Ubuntu Ver. 12.0.4 Operating Systems

3) We have reason to believe that the fault of corrupting the OS was due to our own negligence of attempting to boot the system from a portable device (a pen drive), which may have corrupted the OSs system files.


1) OK. Proceed on to the next phase. Avoid trying to experiment with unnecessary methods.




WEEK : 11 DATE : 17/09/2012 – 21/09/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Reconfigure NFS and SSH in each of the 3 nodes.

2) Establish connectivity in all 3 nodes.

3) Troubleshoot the issue regarding one of the Slave node’s inability to connect to the network.


1) Completed. All of the nodes are configured with the required protocols.

2) Partially complete. One of the server nodes failed to connect to the other devices.

3) Completed. Configured correct IP address.

REFLECTIVE :

1) Each of the nodes required to be reconfigured with NFS and SSH protocols because we had reformatted and reinstalled the OS in each of the Desktop PCs last week.

2) One of the Slave nodes had experienced problems while attempting to connect to the other devices earlier. We later discovered the main cause of this issue was due to an error in assigning static IP addresses.


1) OK. Ensure that the assigned IP addresses are within the same subnet and possess the same Default Gateway address.




WEEK : 12 DATE : 24/09/2012 – 28/09/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Head to the library to examine other examples of final year projects.

2) Install VIM to help in writing configurations in text editors.

3) Install MPICH to help distribute among the 3 nodes.


1) Completed. Noted down important key points of the Project Report documentations.

2) Completed. VIM downloaded and installed on all 3 nodes.

3) Completed. MPICH downloaded and installed on all 3 nodes.

REFLECTIVE :

1) Referred to examples of senior’s Viva project report documentations. Studied the layout of their reports, objectives, the key points, strengths and weaknesses and benchmark result comparisons, which helped us in producing our own documentation.

2) The VIM functions as an alternative to the default text editor application in the Linux OS. Earlier on we had issues using the text editor, and the VIM serves as an alternative method of editing and writing configuration files.

3) The MPICH is an application dedicated to distributing amongst the 3 nodes. This is a particularly essential feature in Parallel Computing, or Server Clustering.


1) Good. You may also refer to the JTMK blog site for more information regarding the layout and formatting of the Project documentation.




WEEK : 13 DATE : 01/10/2012 – 05/10/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Download and install LAM for the MPICH.

2) Configure the Master node to accept LAM for use with the MPICH application.

3) Troubleshoot problem with LAM.


1) Completed.

2) Incomplete. Unable to pair LAM with the MPICH application.

3) Incomplete. Unable to obtain any solutions online.

REFLECTIVE :

1) We were unable to detect the cause of the issue regarding this problem. We will forward this issue to Mr. Assidiq and consult him regarding the problem.


1) OK. Proceed with the current task and find a solution as soon as possible.




WEEK : 14 DATE : 08/10/2012 – 12/10/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Meet up with Mr. Assiddiq to discuss our issue regarding MPI-LAM.

2) Set up Slave nodes with LILO.

3) Modify the etc/lilo.conf file on both Slave nodes.


1) Completed. We did not require LAM to work with our MPICH2 application.

2) Completed.

3) Completed. Both Slave PCs had been configured for LILO.

REFLECTIVE :

1) According to Mr. Assiddiq the MPI application currently being used in our project, the MPICH2, does not require LAM as it already has its own Message Passing Interface within itself. Therefore, there was no need for us to utilize LAM.

2) The Slave nodes required to be configured with LILO, because both the Slave nodes operate based on the Master node. In other words, LILO functions to enable the both the Master and Slave nodes to load in sync.


1) OK. Proceed on to the verification and documentation phase.




WEEK : 15 DATE : 15/10/2012 – 19/10/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Proceed on to the verification process of the prototype.

2) Document the results and produce a User’s guide for the project.

3) Produce a benchmark result and provide comparison with benchmark result(s) from other server setups.


1) Ongoing. Currently running the prototype system and checking for errors and any other possible issues.

2) Complete. Results have been recorded and a Step by Step guide for the documentation has been completed.

3) Incomplete. Awaiting opportunity to obtain benchmark results from the JTMK’s lab server.

REFLECTIVE :

1) Now that most of the configurations have been complete, we are currently in the process of verifying our system to confirm its functionality and that it is in working order as predicted.

2) The benchmark result comparison serves as an indication of what our Clustered Server system is capable of in terms of performance. Our results will be compared against those of a conventional server to provide a clear understanding of the capabilities of the Clustered Server, and also this project.


1) Good. Be sure to also provide other useful data which would help in providing a more detailed explanation of your project’s capabilities.




WEEK : 16 DATE : 22/10/2012 – 25/10/2012 TIME : 10:00 A.M.


WEEKLY PLANNING :

1) Hand in final draft of the Log Book report to project supervisor.

2) Make final adjustments and error checks on edited draft.

3) Submit completed Log Book report and draft of complete documentation.


1) Completed. Handed in final draft on Wednesday.

2) Completed. Printed out completed Log Book report.

3) Completed. Submitted Log Book report to project supervisor for evaluation and marking purposes.

REFLECTIVE :

1) The Log Book has finally been completed and checked.

2) All that’s left to do now is to coordinate and plan our prototype for the upcoming Viva presentation on next Wednesday.


1) Good Job. Concentrate on the overall project documentation and next week’s final Viva presentation.



f5228 beowulf cluster server - log book report

Documents

enclosed detail attachment

low yat plaza

linux ubuntu ver

hazri bin hasnan

linux ubuntu os

year session

port desktop switch

private ssh keys