- LOGZILLA FORMERLY PHP-SYSLOG-NG - █▌ █▌ █▌ █ ▐▌ █ LogZilla is used to integrate syslog-ng messages (syslog-ng tutorial here) ▐▌ █ into a beautiful web interface - demo HERE login/pass is demo/demo. ▐▌ █ LogZilla site is HERE and the forums are HERE with excellent support from ▐▌ █ the author and friends. ▐▌ █ ▐▌ █ There are install instructions HERE that are easy to follow, and If you ▐▌ █ like visual instructions there is an .mp4 HERE. ▐▌ █ ▐▌ █ I decided to write my own mewbies tutorial so that we cover all the ▐▌ █ prerequisites steps first so very little, if any, trouble shooting after. ▐▌ █ There are quite a few things to do, but how nice when you do open your ▐▌ █ LogZilla page for the first time and it all works smoothly :D . This is ▐▌ █ how I did it on my Debian. ▐▌ █ ▐▌ █ PREREQUISITES: ▐▌ █ INSTALL LOGZILLA: ▐▌ █ SPHINX: ▐▌ █ LOG REPLAY VIA CONSOLE: ▐▌ █ TROUBLE SHOOTING: ▐▌ █ MY MYSQL MISC NOTES: ▐▌ █ ▐▌ █ PREREQUISITES: ▐▌ █ `````````````` ▐▌ █ Apache ▐▌ █ MySQL 5.1 or higher ▐▌ █ PHP 4.0.5 or higher with ionCube ▐▌ █ syslog-ng ▐▌ █ build-essential ▐▌ █ gcc ▐▌ █ libapache2-mod-php5 ▐▌ █ libmysqlclient15-dev ▐▌ █ libmysqlclient16 ▐▌ █ msttcorefonts ▐▌ █ php5-cli ▐▌ █ php5-gd ▐▌ █ php5-mysql ▐▌ █ perl ▐▌ █ And perl modules: ▐▌ █ Date::Calc (libdate-calc-perl) ▐▌ █ String::CRC32 (libstring-crc32-per) ▐▌ █ Text::LevenshteinXS ▐▌ █ ▐▌ █ To check if you have those needed, except for Text::LevenshteinXS and ▐▌ █ ionCube - covered after: ▐▌ █ dpkg -l | grep -E 'apache|mysql-*5*|php|libapache*-mod-php' ▐▌ █ Output to look for would be similar to: ▐▌ █ ii apache2 2.2.9-10+lenny8 Apache HTTP Server metap ▐▌ █ ii libapache2-mod-php5 5.2.6.dfsg.1-1+lenny8 server-side, HTML-embedd ▐▌ █ ii libapache2-mod-python 3.3.1-7 Python-embedding module ▐▌ █ ii libapache2-reload-perl 0.10-2 Reload Perl modules when ▐▌ █ ii mysql-client-5.1 5.1.37-1~bpo50+1 MySQL database client ▐▌ █ ii mysql-server-5.1 5.1.37-1~bpo50+1 MySQL database server ▐▌ █ ii php5 5.2.6.dfsg.1-1+lenny8 server-side, HTML-embedd ▐▌ █ ii php5-cli 5.2.6.dfsg.1-1+lenny8 command-line interpreter ▐▌ █ ii php5-gd 5.2.6.dfsg.1-1+lenny8 GD module for php5 ▐▌ █ ii php5-mysql 5.2.6.dfsg.1-1+lenny8 MySQL module for php5 ▐▌ █ And: ▐▌ █ dpkg -l build-essential gcc libdate-calc-perl libmysqlclient15-dev libmysqlclient16 libstring-crc32-perl msttcorefonts perl syslog-ng █ ii build-essential 11.4 Informational list of ▐▌ █ ii gcc 4:4.3.2-2 The GNU C compiler ▐▌

█ ii libdate-calc-perl 5.4-5+b1 Perl library for ▐▌ █ ii libmysqlclient15-dev 5.0.51a-24+lenny4 MySQL database ▐▌ █ ii libmysqlclient16 5.1.37-1~bpo50+1 MySQL database client ▐▌ █ ii libstring-crc32-perl 1.4-2+b1 Perl interface for cycli ▐▌ █ ii msttcorefonts 2.7 transitional dummy ▐▌ █ ii perl 5.10.0-19lenny2 Larry Wall's Practical ▐▌ █ ii syslog-ng 2.0.9-4.1 Next generation logging ▐▌ █ ▐▌ █ PREREQUISITES INSTALL NOTES: ▐▌ █ -For MySQL 5.1 on Debian we'll do that last, and yes you must have 5.1 or ▐▌ █ higher. ▐▌ █ ▐▌ █ -Module Text::LevenshteinXS will be installed using 'cpan' - shown how ▐▌ █ after. ▐▌ █ ▐▌ █ INSTALL: ▐▌ █ Add/change the aptitude install list with the programs you need. For ▐▌ █ example, the items I needed to install: ▐▌ █ su ▐▌ █ aptitude update ▐▌ █ aptitude install libmysqlclient-dev libmysqlclient15-dev libdate-calc-perl libstring-crc32-perl msttcorefonts █ *Note: selecting "libmysqlclient15-dev" instead of the ▐▌ █ virtual package "libmysqlclient-dev" ▐▌ █ ▐▌ █ INSTALL PERL MODULE Text::LevenshteinXS: ▐▌ █ To view what perl modules you have installed: ▐▌ █ instmodsh ▐▌ █ l ▐▌ █ q ▐▌ █ Example output: ▐▌ █ Installed modules are: ▐▌ █ Crypt::ircBlowfish ▐▌ █ POE::Component::Client::FTP ▐▌ █ POE::Filter::Ls ▐▌ █ Perl ▐▌ █ ▐▌ █ To install the perl module Text::LevenshteinXS you need to use cpan (it's ▐▌ █ not on apt-get or aptitude yet)- ▐▌ █ cpan -i Text::LevenshteinXS ▐▌ █ Hit enter key to all questions to select its default, then when you come ▐▌ █ to: ▐▌ █ Select your continent (or several nearby continents) [] ▐▌ █ You need to enter a number from the list presented and a few more location ▐▌ █ selections. ▐▌ █ Once it begins to install the output will be similar to: ▐▌ █ [snip] ▐▌ █ Installing ▐▌ █ /usr/local/lib/perl/5.10.0/auto/Text/LevenshteinXS/LevenshteinXS.bs ▐▌ █ [snip] ▐▌ █ JGOLDBERG/Text-LevenshteinXS-0.03.tar.gz ▐▌ █ make install -- OK ▐▌ █ Warning (usually harmless): 'YAML' not installed, will not store ▐▌ █ persistent state ▐▌ █ ▐▌ █ Side note: Another method to view your installed modules with install ▐▌ █ dates and paths: ▐▌ █ perldoc perllocal ▐▌ █ q ▐▌ █ ▐▌ █ INSTALL IONCUBE FOR PHP: ▐▌ █ View my 'IONCUBE & ZEND ENCODED PHP FILES' tutorial under 'WEB SERVERS' ▐▌ █ HERE how to install ionCube. Scroll down to 'PREREQUISITES FOR IONCUBE ▐▌ █ ENCODED FILES:' ▐▌

█ After you have confirmed you have ionCube, return here and continue. ▐▌ █ ▐▌ █ If you are using PHP5.3 view HERE. ▐▌ █ ▐▌ █ DEBIAN MYSQL 5.0 UPGRADE TO 5.1: ▐▌ █ This is how I did it successfully and easily for my setup which installed ▐▌ █ 5.1.37-1~bpo50+1. ▐▌ █ For 5.1.48-1 and others view HERE. ▐▌ █ ▐▌ █ 1. To upgrade MySQL from 5.0.x to 5.1 on Debian successfully you'll need ▐▌ █ to comment out a line on your 'my.cnf' file - normally in your home dir or ▐▌ █ /etc/my.cnf or /etc/mysql/my.cnf ▐▌ █ I didn't have this in my home, nor that line in my /etc/mysql/my.cnf ▐▌ █ If you have this line comment it out (place a # in front of the line) or ▐▌ █ just remove the line: skip-bdb ▐▌ █ There might be other issues related to upgrading MySQL so I recommend you ▐▌ █ first read up before proceeding. ▐▌ █ ▐▌ █ 2. At the date of writing this you can not use apt-get install ▐▌ █ mysql-server to install 5.1. ▐▌ █ Read HERE where I acquired the lenny-backports URL, 'Semi-official ▐▌ █ backports of the Debian MySQL packages'. ▐▌ █ ▐▌ █ You'll need to add sources.list the lenny-backports: ▐▌ █ pico /etc/apt/sources.list ▐▌ █ Add this at the bottom of the file: ▐▌ █ deb http://people.debian.org/~nobse/mysql-dfsg-5.1/ lenny-backports main ▐▌ █ ▐▌ █ 3. Then proceed with the upgrade: ▐▌ █ aptitude update ▐▌ █ aptitude -P -t lenny-backports install mysql-server-5.1 ▐▌ █ Answer Yes to all questions. ▐▌ █ ▐▌ █ 4. After the upgrade is finished you need to upgrade your databases. ▐▌ █ First: ▐▌ █ chmod 666 /var/lib/mysql/mysql_upgrade_info ▐▌ █ Or the upgrade will give the error "Could not create the upgrade info ▐▌ █ file '/var/lib/mysql/mysql_upgrade_info' in the MySQL Servers datadir, ▐▌ █ errno: 13" ▐▌ █ Then run the upgrade: ▐▌ █ mysql_upgrade -u root -p --verbose --force ▐▌ █ Output might be similar to: ▐▌ █ mysql.general_log ▐▌ █ Error : You can't use locks with log tables. ▐▌ █ status : OK ▐▌ █ mysql.slow_log ▐▌ █ Error : You can't use locks with log tables. ▐▌ █ status : OK ▐▌ █ [snip] ▐▌ █ Running 'mysql_fix_privilege_tables'... ▐▌ █ OK ▐▌ █ ▐▌ █ These lock errors you can ignore; there are more of a message and ▐▌ █ supposedly will be removed in future versions, read HERE. ▐▌ █ ▐▌ █ Restart mysql: ▐▌ █ /etc/init.d/mysql restart ▐▌ █ ▐▌ █ MYSQL SETTINGS: ▐▌ █ For this version you need to increase the thread_stack, read HERE. These ▐▌ █ errors would be present in /var/log/daemon.log "mysqld: ... Thread stack ▐▌ █ overrun: 3971 bytes used of a 131072 byte stack, and 128000 bytes needed. ▐▌ █ Use 'mysqld -O thread_stack=#' to specify a bigger stack" ▐▌ █ ▐▌

█ To fix this change the setting in 'my.cnf' /etc/mysql/my.cnf or yours ▐▌ █ might be here /etc/my.cnf: ▐▌ █ pico /etc/mysql/my.cnf ▐▌ █ Has this: ▐▌ █ thread_stack = 128K ▐▌ █ Change to: ▐▌ █ thread_stack = 256K ▐▌ █ ▐▌ █ Next in my.cnf event scheduler must be set to ON ▐▌ █ Check if you have the code 'event_scheduler=' first, if not add it toward ▐▌ █ the bottom of the file (mine didn't have this directive at all): ▐▌ █ event_scheduler=ON ▐▌ █ ▐▌ █ Save your changes ▐▌ █ ▐▌ █ Then turn on even scheduler in MySQL: ▐▌ █ mysql -u root -p ▐▌ █ SELECT @@event_scheduler; ▐▌ █ ▐▌ █ My output stated off: ▐▌ █ +-------------------+ ▐▌ █ | @@event_scheduler | ▐▌ █ +-------------------+ ▐▌ █ | OFF | ▐▌ █ +-------------------+ ▐▌ █ 1 row in set (0.00 sec) ▐▌ █ ▐▌ █ To turn it on: ▐▌ █ SET GLOBAL event_scheduler = 1; ▐▌ █ ▐▌ █ Output then stated: ▐▌ █ Query OK, 0 rows affected (0.00 sec) ▐▌ █ SELECT @@event_scheduler; ▐▌ █ Output now states: ▐▌ █ +-------------------+ ▐▌ █ | @@event_scheduler | ▐▌ █ +-------------------+ ▐▌ █ | ON | ▐▌ █ +-------------------+ ▐▌ █ 1 row in set (0.00 sec) ▐▌ █ ▐▌ █ exit; ▐▌ █ ▐▌ █ Then restart MySQL: ▐▌ █ /etc/init.d/syslog-ng restart ▐▌ █ ▐▌ █ Note: If your event scheduler isn't on then during installation of ▐▌ █ LogZilla you'll receive this error: ▐▌ █ DBD::mysql::st execute failed: Cannot proceed because system tables used ▐▌ █ by Event Scheduler were found damaged at server start at ./install.pl line ▐▌ █ 411, <STDIN> line 17. ▐▌ █ ▐▌ █ SSYSLOG-NG SETTINGS: ▐▌ █ pico /etc/syslog-ng/syslog-ng.conf ▐▌ █ Search for (Ctrl+w): udp(); ▐▌ █ If it is commented out: ▐▌ █ # udp(); ▐▌ █ Remove the comment; change to: ▐▌ █ udp(); ▐▌ █ ▐▌ █ Then restart syslog-ng: ▐▌ █ /etc/init.d/syslog-ng restart ▐▌ █ ▐▌ █ PHP SETTINGS: ▐▌

█ We also need to have set for PHP a 'maximum execution time' of a minimum ▐▌ █ of 300 seconds and the 'memory limit' of at least 128MB : ▐▌ █ pico /etc/php5/apache2/php.ini ▐▌ █ Search for (Ctrl+w): max_execution_time ▐▌ █ Mine has (my execution time is already set very high for other reasons): ▐▌ █ max_execution_time = 9600 ▐▌ █ memory_limit = -1 ▐▌ █ I changed it to: ▐▌ █ max_execution_time = 9600 ▐▌ █ memory_limit = 128M ▐▌ █ ▐▌ █ Restart web server: ▐▌ █ /etc/init.d/apache2 restart ▐▌ █ exit ▐▌ █ ▐▌ █ INSTALL LOGZILLA: ▐▌ █ ````````````````` ▐▌ █ Finally, we are ready to start working on LogZilla installation. ▐▌ █ Check the site for the latest version. For this tutorial I've used 3.0.85: ▐▌ █ wget http://php-syslog-ng.googlecode.com/files/logzilla_3.0.85.tgz ▐▌ █ su ▐▌ █ cp /home/user/logzilla_3.0.85.tgz /var/www/logzilla_3.0.85.tgz ▐▌ █ cd /var/www/ ▐▌ █ tar xvzf logzilla_3.0.85.tgz ▐▌ █ ▐▌ █ My own preference when dealing with sensitive information on web servers ▐▌ █ is to rename the default install path. So for logzilla I'll rename it to ▐▌ █ for example 'lz' and I'll refer to path 'logzilla' as lz for the rest of ▐▌ █ this tutorial: ▐▌ █ mv /var/www/logzilla/ /var/www/lz/ ▐▌ █ rm logzilla_3.0.85.tgz ▐▌ █ cat /var/www/lz/README ▐▌ █ ▐▌ █ Note about license key, read HERE & HERE, "Copy will simply stop working ▐▌ █ (the web interface portion). At that point, you can either download the ▐▌ █ new version or a new license key." ▐▌ █ cat /var/www/lz/license.txt ▐▌ █ Expires: 12 Sep 2010 ▐▌ █ ▐▌ █ For LogZilla v3.0.85 only, remove the facebox call as it is not being used ▐▌ █ from this v + as stated HERE: ▐▌ █ pico /var/www/lz/html/includes/css.php ▐▌ █ Remove this section: ▐▌ █ <!-- BEGIN Facebox --> ▐▌ █ <link type="text/css" rel="stylesheet" ▐▌ █ href="includes/js/jquery/plugins/facebox/facebox.css" /> ▐▌ █ <!-- END Facebox --> ▐▌ █ ▐▌ █ PRE INSTALL NOTES: ▐▌ █ A. LogZilla comes with an installer- perl script. You'll be asked a number ▐▌ █ of questions, to select the default answer in [brackets] just hit your ▐▌ █ enter key, otherwise type in your own setting. ▐▌ █ ▐▌ █ B. Below are the questions you'll be asked and the answers I used; if I ▐▌ █ haven't typed an answer for the question it's because I hit enter key to ▐▌ █ select the answer provided in [brackets]. ▐▌ █ ▐▌ █ C. The 3rd from last question "Enter the base url for your site (include ▐▌ █ trailing slash) [/logs/]" - This path is what you will use in your apache ▐▌ █ settings as an 'alias'. This means that if your answer is /logs/ your URL ▐▌ █ to go to /var/www/logzilla/ would be http://yoursite.com/logs/ . Another ▐▌ █ example would be the answer / which then logzilla install would be reached ▐▌ █ by going to http://yoursite.com ▐▌ █ ▐▌

█ D. If you want to change the answer to one of your questions in the first ▐▌ █ section 'before the install is complete', just quit the installer and ▐▌ █ start over. ▐▌ █ If you need to change anything in the install once it is finished see ▐▌ █ under 'TROUBLE SHOOTING' - 'REINSTALL LOGZILLA FRESH'. ▐▌ █ ▐▌ █ INSTALL: ▐▌ █ OK lets do it: ▐▌ █ First make a copy of your 'syslog-ng.conf' just in case you want to ▐▌ █ install LogZilla fresh: ▐▌ █ cp /etc/syslog-ng/syslog-ng.conf /etc/syslog-ng/syslog-ng-pre_logzilla.conf▐▌ █ ▐▌ █ cd /var/www/lz/scripts/ ▐▌ █ Run the installer: ▐▌ █ ./install.pl ▐▌ █ ▐▌ █ Enter the MySQL root username [root]: ▐▌ █ Note: Mysql passwords with a ' in them may not work ' ▐▌ █ Enter the password for root [mysql]: MyRootMysqlPass ▐▌ █ Database to install to [syslog]: lz ▐▌ █ Database table to install to [logs]: ▐▌ █ Enter the name of the MySQL server [127.0.0.1]: localhost ▐▌ █ Enter the port of the MySQL server [3306]: ▐▌ █ Enter the name to create as the owner of the logs database [syslogadmin]: ▐▌ █ Enter the password for the syslogadmin user [syslogadmin]: SysLogPass ▐▌ █ Enter the name to create as the WEBSITE owner [admin]: ▐▌ █ Enter the password for admin [admin]: AdminPass ▐▌ █ Enter your email address [cdukes@cdukes.com]: mine@email.com ▐▌ █ Enter a name for your website [The home of LogZilla]: LZS ▐▌ █ Enter the base url for your site (include trailing slash) [/logs/]: ▐▌ █ Where should log files be stored? [/var/log/logzilla]: ▐▌ █ How long should I keep old logs? (in days) [30]: ▐▌ █ ▐▌ █ Path Updates: ▐▌ █ Getting ready to replace paths in all files with "/var/www/lz" ▐▌ █ Ok to continue? [y]: ▐▌ █ Updating file paths ▐▌ █ Updating log paths ▐▌ █ ▐▌ █ Database Installation: ▐▌ █ All data will be installed into the lz database ▐▌ █ Ok to continue? [y]: ▐▌ █ ▐▌ █ Config.php generation: ▐▌ █ Generating /var/www/lz/html/config/config.php ▐▌ █ Ok to continue? [y]: ▐▌ █ ▐▌ █ System files: ▐▌ █ Adding LogZilla logrotate.d file to /etc/logrotate.d ▐▌ █ Ok to continue? [y]: ▐▌ █ Where is your syslog-ng.conf file located? ▐▌ █ [/etc/syslog-ng/syslog-ng.conf]: ▐▌ █ Adding syslog-ng configuration to /etc/syslog-ng/syslog-ng.conf ▐▌ █ Ok to continue? [y]: ▐▌ █ Found 1 sources ▐▌ █ Which source definition would you like to use? [s_all]: ▐▌ █ ▐▌ █ LogZilla installation complete... ▐▌ █ ▐▌ █ POST INSTALL NOTES: ▐▌ █ During the 'Config.php generation:' step during install it stated ▐▌ █ "Generating /var/www/logzilla/html/config/config.php" ▐▌ █ This will be removed after by the installer: ▐▌ █ ls -al /var/www/ ▐▌

█ ▐▌ █ LogZilla logs are rotated daily (by this file /etc/logrotate.d/logzilla) ▐▌ █ and the logs will be 0 bytes unless debugging is enabled: ▐▌ █ ls -al /var/log/logzilla ▐▌ █ ▐▌ █ TEST MYSQL LOGZILLA USER CAN LOGIN: ▐▌ █ Test that the new user, syslogadmin, you created during install can login, ▐▌ █ change 'lz' below to the name of your database (default syslog): ▐▌ █ mysql -u syslogadmin -p lz ▐▌ █ If you logged in fine: ▐▌ █ exit; ▐▌ █ If you receive the error "ERROR 1045 (28000): Access denied for user ▐▌ █ 'syslogadmin'@'localhost' (using password: YES)" ▐▌ █ To fix this, replace PASSWORD below to password you selected for ▐▌ █ syslogadmin and name of data base (lz) IF you changed it during the ▐▌ █ install: ▐▌ █ mysql -u root -p mysql ▐▌ █ GRANT ALL PRIVILEGES ON lz.* TO syslogadmin@localhost IDENTIFIED BY "PASSWORD"; █ FLUSH PRIVILEGES; ▐▌ █ exit; ▐▌ █ ▐▌ █ Test the user login again: ▐▌ █ mysql -u syslogadmin -p lz ▐▌ █ exit; ▐▌ █ ▐▌ █ ADD LOGZILLA TO WEB SERVER: ▐▌ █ pico /etc/apache2/apache2.conf ▐▌ █ First if you don't have your 'ServerName' and your ServerRoot' directives ▐▌ █ set you need to add it, at the top of the file after all the comments (#). ▐▌ █ For example, mine already has it: ▐▌ █ [snip] ▐▌ █ # Do NOT add a slash at the end of the directory path. ▐▌ █ # ▐▌ █ ServerName "mystite.com" ▐▌ █ ServerRoot "/etc/apache2" ▐▌ █ Options -Indexes FollowSymLinks Includes ExecCGI ▐▌ █ [snip] ▐▌ █ ▐▌ █ Then you need to create a vhost file LogZilla, name file how you like, ▐▌ █ again I use lz: ▐▌ █ pico /etc/apache2/sites-available/lz ▐▌ █ Paste this in, changing '/logs' to the path you used during installation ▐▌ █ for the alias and the path to your logzilla/html/: ▐▌# LogZilla Alias /logs "/var/www/lz/html/" Options Indexes MultiViews FollowSymLinks AllowOverride All Order allow,deny Allow from all █ ▐▌ █ Enable it: ▐▌ █ a2ensite lz ▐▌ █ ▐▌ █ Reload apache: ▐▌ █ /etc/init.d/apache2 reload ▐▌ █ ▐▌ █ Now finally give LogZilla a test: http://yoursite.com/logs/login.php ▐▌ █ Login using admin username/password set during install.pl ▐▌ █ ▐▌ █ IONCUBE: ▐▌ █ Just to check all is fine with our ionCube installation, go to: ▐▌

█ http://yoursite.com/logs/ioncube/ ▐▌ █ If all is fine it will state: ▐▌ █ ionCube Loader Wizard ▐▌ █ The ionCube Loader 3.3.17 is already installed [snip] ▐▌ █ ▐▌ █ If all is well then press back button on your browser, then in your shell: ▐▌ █ rm /var/www/lz/html/ioncube -rf ▐▌ █ rm /var/www/lz/html/ioncube_64 -rf ▐▌ █ ▐▌ █ DEDUPLICATION: ▐▌ █ From the User Guide: "When Deduplication is enabled, "similar" messages ▐▌ █ are rolled up into a single message. When that happens, the FO, LO and ▐▌ █ Counter columns get updated." ▐▌ █ "If deduplication is disabled, the FO and LO columns are identical." ▐▌ █ So to enable this, on your LogZilla page go to Admin's Options/Server ▐▌ █ Admin, click on the blue button left column and select 1 to enable it. ▐▌ █ ▐▌ █ SPHINX: ▐▌ █ ``````` ▐▌ █ INSTALL: ▐▌ █ From the User Guide: "Sphinx is used to provide the full text search ▐▌ █ capabilities for LogZilla, it is an order of magnitude faster than using ▐▌ █ MySQL alone (much, much faster)." ▐▌ █ ▐▌ █ Note that during the installing process of LogZilla it will have updated ▐▌ █ the /sphinx/sphinx.conf file (Modifying ../sphinx/sphinx.conf) to your ▐▌ █ correct logzilla install paths so that you don't need to edit this file. ▐▌ █ To install Sphinx: ▐▌ █ cd /var/www/lz/sphinx/src ▐▌ █ tar xzvf sphinx-0.9.9.tar.gz ▐▌ █ cd sphinx-0.9.9 ▐▌ █ Compile it: ▐▌ █ ./configure --prefix `pwd`/../.. ▐▌ █ Output: A long list of checking etc will scroll by, then it will end with: ▐▌ █ You can now run 'make' to build Sphinx binaries, ▐▌ █ and then run 'make install' to install them. ▐▌ █ ▐▌ █ First check if there are any serious errors such as 'ERROR: cannot find ▐▌ █ MySQL include files.' ▐▌ █ If so it's most likely that you don't have all the prerequisites. ▐▌ █ If you did have a serious error don't continue to the next step, fix the ▐▌ █ error/s, and run the ./configure cmd again before proceeding. ▐▌ █ View HERE for my list of negatives 'no' the ./configure output gave to me ▐▌ █ and how I fixed those. None will hinder Sphinx for LogZilla, that I know ▐▌ █ of, only for other programs you might want to use Sphinx on. ▐▌ █ ▐▌ █ After you have the output you would like on the sphinx configure, run: ▐▌ █ make && make install ▐▌ █ Output - you'll have a long pause followed by a long install log: ▐▌ █ Making all in src ▐▌ █ make[1]: Entering directory `/var/www/lz/sphinx/src/sphinx-0.9.9/src' ▐▌ █ [snip] ▐▌ █ make[1]: Leaving directory `/var/www/lz/sphinx/src/sphinx-0.9.9' ▐▌ █ ▐▌ █ CREATE YOUR FIRST INDEX: ▐▌ █ cd /var/www/lz/sphinx ▐▌ █ ./indexer.sh full ▐▌ █ Output: ▐▌ █ Starting Sphinx Indexer: 2010-07-25 12:41:47 ▐▌ █ No previous index files found ▐▌ █ Creating NEW indexes, this may take a while, so be patient... ▐▌ █ Running command: /var/www/lz/sphinx/bin/indexer --config ▐▌ █ /var/www/lz/sphinx/sphinx.conf idx_logs idx_delta_logs ▐▌ █ Sphinx 0.9.9-release (r2117) ▐▌

█ [snip] ▐▌ █ Finished Sphinx Indexer: 2010-07-25 12:41:48 ▐▌ █ ▐▌ █ NOTE: If you see the error when creating the full index: ▐▌ █ "ERROR: index 'idx_logs': sql_query_pre[0]: Column 'max_id' cannot be null ▐▌ █ (DSN=mysql://syslogadmin:***@127.0.0.1:3306/syslog)." ▐▌ █ This means the database doesn't have your log entries yet, wait a bit then ▐▌ █ try again. ▐▌ █ ▐▌ █ Test the other indexer commands that we'll use on crontab to be sure no ▐▌ █ errors: ▐▌ █ ./indexer.sh delta ▐▌ █ ./indexer.sh merge ▐▌ █ ▐▌ █ START SPHINX SEARCH DAEMON: ▐▌ █ Start sphinx search daemon 'searchd': ▐▌ █ bin/searchd ▐▌ █ Output: ▐▌ █ [snip] ▐▌ █ listening on 127.0.0.1:9312 ▐▌ █ listening on all interfaces, port=3312 ▐▌ █ ▐▌ █ ps x ▐▌ █ 19613 ? S 0:00 bin/searchd ▐▌ █ After a reboot it will look like this: ▐▌ █ 3217 ? S 0:03 /var/www/lz/sphinx/bin/searchd -c /var/www/lz/sphinx/sphinx.conf █ ▐▌ █ SPHINX INDEXER AUTO RUN: ▐▌ █ A crontab file is included in ▐▌ █ scripts/contrib/system_configs/logzilla.crontab. Read it to explain the ▐▌ █ settings: ▐▌ █ cat /var/www/lz/scripts/contrib/system_configs/logzilla.crontab ▐▌ █ Line 1: 12:30 AM on the first of every month it will run full ▐▌ █ Line 2: Every 5 mins of every day it will run delta ▐▌ █ Line 3: 12:00 AM every day it will run merge ▐▌ █ To add these: ▐▌ █ crontab -e ▐▌ █ Paste in at the bottom:, changing to your correct paths - 2 paths per line:▐▌30 0 1 * * /var/www/lz/sphinx/indexer.sh full >> /var/log/logzilla/sphinx_indexer.log 2>&1*/5 * * * * /var/www/lz/sphinx/indexer.sh delta >> /var/log/logzilla/sphinx_indexer.log 2>&10 0 * * * /var/www/lz/sphinx/indexer.sh merge >> /var/log/logzilla/sphinx_indexer.log 2>&1 █ ▐▌ █ Note: ▐▌ █ Before crontab runs those cmds you do not have the file ▐▌ █ 'sphinx_indexer.log', after it runs the first time (within 5 mins) it will ▐▌ █ create it: ▐▌ █ ls -al /var/log/logzilla ▐▌ █ BTW this log will be also be rotated by /etc/logrotate.d/logzilla as it ▐▌ █ declares *.log. ▐▌ █ ▐▌ █ SPHINX SEARCH DAEMON AUTO RUN AFTER REBOOT: ▐▌ █ pico /etc/rc.local ▐▌ █ Paste in just before the last line 'exit 0' changing the paths to your ▐▌ █ own: ▐▌ █ /var/www/lz/sphinx/bin/searchd -c /var/www/lz/sphinx/sphinx.conf ▐▌ █ ▐▌ █ SPHINX LOG ROTATION: ▐▌ █ ls -al /var/www/lz/sphinx/log ▐▌ █ I didn't find a log rotation setup for it, nor any other logs: ▐▌ █ grep -r sphinx* /etc/* ▐▌

█ find / -name *query.log ▐▌ █ ▐▌ █ So following the information HERE, I'll set up log rotation for those by ▐▌ █ doing this: ▐▌ █ Take note of the perms, owner and group of the logs as we'll use the same ▐▌ █ for our logrotate config. ▐▌ █ Mine are 600 root root (-rw------- 1 root root) ▐▌ █ Create the log rotation configuration file: ▐▌ █ pico /etc/logrotate.d/sphinx ▐▌ █ Paste in, changing to your correct paths and preferences: ▐▌/var/www/lz/sphinx/log/*.log { weekly notifempty rotate 3 compress delaycompress create 600 root root postrotate killall -SIGUSR1 searchd endscript} █ ▐▌ █ To test, but NOT actually rotate sphinx logs: ▐▌ █ logrotate --debug --force /etc/logrotate.d/sphinx ▐▌ █ ▐▌ █ To rotate the logs manually: ▐▌ █ logrotate --force -v /etc/logrotate.d/sphinx ▐▌ █ ls -al /var/www/lz/sphinx/log ▐▌ █ ▐▌ █ Done :D Have a look at the User Guide HERE. ▐▌ █ For large servers read HERE how to increase your UDP buffer size. ▐▌ █ ▐▌ █ LOG REPLAY VIA CONSOLE: ▐▌ █ ``````````````````````` ▐▌ █ This will allow you to replay a log file taken from another server. ▐▌ █ Included is a sample log (syslog.sample.gz) for you test logreplay with ▐▌ █ it: ▐▌ █ cd /var/www/lz/scripts/contrib/logreplay ▐▌ █ This will extract the 3MB sample 'syslog.sample' and delete the pack ▐▌ █ (syslog.sample.gz): ▐▌ █ gzip -d syslog.sample.gz ▐▌ █ ▐▌ █ To use this we need to compile the included spoof program so that it "will ▐▌ █ rewrite the outgoing syslog packet and insert the hostnames from the ▐▌ █ syslog.sample file so that when syslog-ng receives the messages they ▐▌ █ appear to come from that host instead of your local machine.": ▐▌ █ gcc spoof_syslog.c -o spoof_syslog ▐▌ █ There shouldn't be any output/reply after running that cmd. ▐▌ █ Doing this created the binary spoof_syslog used by logreplay.pl ▐▌ █ ls -al ▐▌ █ -rwxr-xr-x 1 root root 9357 2010-07-25 08:13 spoof_syslog ▐▌ █ Set executable perms on logreplay.pl script: ▐▌ █ chmod +x logreplay.pl ▐▌ █ Run the scripts help menu: ▐▌ █ ./logreplay.pl -h ▐▌ █ Play the sample log: ▐▌ █ ./logreplay.pl -v -f ./syslog.sample -s ./spoof_syslog ▐▌ █ Output will end with: ▐▌ █ Host limit of 50 reached, use ./logreplay.pl -l to set a higher limit ▐▌ █ Sent 123 messages out ▐▌ █ ▐▌ █ TROUBLE SHOOTING: ▐▌ █ ````````````````` ▐▌ █ 1. Best place is the LogZilla forum HERE. ▐▌

█ ▐▌ █ 2. If you aren't sure the behaviour of your LogZilla page is correct I ▐▌ █ test it by doing the same search on the Demo HERE (demo:demo). ▐▌ █ ▐▌ █ 3. Turn on debugging via LogZilla page, top link Admin /Server Admin, ▐▌ █ page 2, click on the blue marble left column. Once this is on it will show ▐▌ █ the debug information on every page and start writing to the file ▐▌ █ /var/log/logzilla/db_insert.log . Check that file for errors: ▐▌ █ tail -50 /var/log/logzilla/db_insert.log ▐▌ █ Remember to turn debugging off as this file can grow large very fast. If ▐▌ █ you have left it on, not to worry as the LogZilla's log rotation will ▐▌ █ rotate it daily (/etc/logrotate.d/logzilla). ▐▌ █ ▐▌ █ 4. Take a look at your logs for errors related: ▐▌ █ tail -20 /var/log/syslog ▐▌ █ cat /var/log/syslog | grep mysqld ▐▌ █ cat /var/log/syslog | grep ERROR ▐▌ █ cat /var/log/syslog | grep crashed ▐▌ █ ▐▌ █ tail -20 /var/log/daemon.log ▐▌ █ cat /var/log/daemon.log | grep ERROR ▐▌ █ ▐▌ █ If you see: ▐▌ █ [Note] Plugin 'FEDERATED' is disabled. ▐▌ █ Not to worry as the federated engine was disabled in v 5.1 ▐▌ █ ▐▌ █ If you see: ▐▌ █ [Warning] Event Scheduler: [root@localhost][lz.logs_del_partition] No data █ - zero rows fetched, selected, or processed █ This is just a message stating that there is nothing to delete; don't █ worry about it, view HERE. █ ▐▌ █ If you have errors similar to (replace lz with your database name): ▐▌ █ [ERROR] /usr/sbin/mysqld: Table './lz/settings' is marked as crashed and should be repaired █ [Warning] Checking table: './lz/settings' ▐▌ █ ▐▌ █ Then you should repair your database and/or tables: ▐▌ █ mysqlcheck -u syslogadmin -p --auto-repair -A ▐▌ █ Or as root to check all database tables: ▐▌ █ mysqlcheck -uroot -p -A -a -c -o -g --auto-repair ▐▌ █ This will analyze, check, optimize, upgrade if needed and repair. ▐▌ █ Or if you want to check all your databases: ▐▌ █ mysqlcheck -uroot -p --check --all-databases --auto-repair ▐▌ █ ▐▌ █ Once that is done, to check your tables (replace lz with your database ▐▌ █ name): ▐▌ █ mysqlcheck -uroot -p lz --check ▐▌ █ mysql -u root -p lz ▐▌ █ Show tables; ▐▌ █ CHECK TABLE banned_ips, cache, facilities, groups, help, history, hosts, logs, logs_archive, lzecs; █ CHECK TABLE mne, programs, settings, severities, sph_counter, suppress, totd, ui_layout, users; █ ▐▌ █ 5. If logging into your LogZilla page you receive the error: ▐▌ █ Error in perform_query function ▐▌ █ No DB link for query: SELECT name,value, type FROM settings ▐▌ █ Mysql_error: Access denied for user 'syslogadmin'@'localhost' (using ▐▌ █ password: YES) ▐▌ █ Refer to earlier step above - 'TEST MYSQL LOGZILLA USER CAN LOGIN:' ▐▌ █ ▐▌ █ 6. If doing a search you receive the error "Warning: Wrong parameter count ▐▌ █ for max() in ▐▌

█ /var/www/lz/html/includes/portlets/portlet-chart_adhoc.php on line 453" ▐▌ █ This only means that the search didn't yield results, view HERE. ▐▌ █ ▐▌ █ 7. Change the URL alias or base path: ▐▌ █ If you need to change the alias for your LogZilla, login to the MySQL ▐▌ █ database and change /logs/ below to the alias you would like: ▐▌ █ mysql -u syslogadmin -p lz ▐▌ █ update settings set value='/logs/' where name='SITE_URL'; ▐▌ █ Output will be: ▐▌ █ Query OK, 1 row affected (0.00 sec) ▐▌ █ Rows matched: 1 Changed: 1 Warnings: 0 ▐▌ █ exit; ▐▌ █ ▐▌ █ Then change your alias in apache: ▐▌ █ pico /etc/apache2/sites-available/lz ▐▌ █ /etc/init.d/apache2 reload ▐▌ █ ▐▌ █ Reopen your browser if changes aren't showing. ▐▌ █ ▐▌ █ If you need to change the path base: ▐▌ █ mysql -u syslogadmin -p lz ▐▌ █ update settings set value='/var/www/html/lz' where name='PATH_BASE'; ▐▌ █ Query OK, 1 row affected (0.01 sec) ▐▌ █ Rows matched: 1 Changed: 1 Warnings: 0 ▐▌ █ ▐▌ █ 8. REINSTALL LOGZILLA FRESH: ▐▌ █ If for some reason you want to start with a fresh LogZilla install you'll ▐▌ █ need to do a few simple steps: ▐▌ █ A. SYSLOG-NG SETTINGS: ▐▌ █ Restore syslog-ng.conf to pre-logzilla install state: ▐▌ █ IF you didn't make a copy of syslog-ng.conf prior to installing LogZilla: ▐▌ █ pico /etc/syslog-ng/syslog-ng.conf ▐▌ █ At the bottom of the file are LogZilla added settings from: ▐▌ █ ########################################################################## ▐▌ █ # Clay's LogZilla config below ▐▌ █ To: ▐▌ █ # Clay's LogZilla config above ▐▌ █ ########################################################################## ▐▌ █ Remove the entire section. ▐▌ █ ▐▌ █ IF you made a back up copy of syslog-ng.conf prior to installing LogZilla: ▐▌ █ rm /etc/syslog-ng/syslog-ng.conf ▐▌ █ mv /etc/syslog-ng/syslog-ng-pre_logzilla.conf /etc/syslog-ng/syslog-ng.conf▐▌ █ ▐▌ █ Then in both cases: ▐▌ █ /etc/init.d/syslog-ng restart ▐▌ █ ▐▌ █ B. MYSQL DATABASE AND USERS: ▐▌ █ Note that I have named my database 'lz', change below to the name of your ▐▌ █ database: ▐▌ █ mysql -u root -p ▐▌ █ DROP DATABASE lz; ▐▌ █ FLUSH PRIVILEGES; ▐▌ █ USE mysql; ▐▌ █ Then to remove the user/s added: ▐▌ █ SELECT User,Password,Host from user; ▐▌ █ DELETE FROM user WHERE user='syslogadmin'; ▐▌ █ SELECT User,Password,Host from user; ▐▌ █ show databases; ▐▌ █ FLUSH PRIVILEGES; ▐▌ █ exit; ▐▌ █ ▐▌ █ C. During the install process it states ▐▌ █ "Adding LogZilla logrotate.d file to /etc/logrotate.d" ▐▌

█ It will NOT make a double file and or entry, so you can leave this file as ▐▌ █ is: /etc/logrotate.d/logzilla ▐▌ █ ▐▌ █ Done. Restart your install with ./install.pl ▐▌ █ ▐▌ █ MY MYSQL MISC NOTES: ▐▌ █ ```````````````````` ▐▌ █ Login: ▐▌ █ mysql -u root -p ▐▌ █ Logout: ▐▌ █ exit; ▐▌ █ Select a database, for example lz: ▐▌ █ USE lz; ▐▌ █ Login selecting lz database: ▐▌ █ mysql -u root -p lz ▐▌ █ Or as the LogZilla user: ▐▌ █ mysql -u syslogadmin -p lz ▐▌ █ View LogZilla database settings; ▐▌ █ SELECT name,value, type FROM settings; ▐▌ █ View users: ▐▌ █ USE mysql; ▐▌ █ SELECT User,Password,Host from user; ▐▌ █ Change admin pass for LogZilla: ▐▌ █ update users set pwhash=md5('MYNEWPASSWORD') where username='admin'; ▐▌ █ Delete a user: ▐▌ █ DELETE FROM user WHERE user='syslogadmin'; ▐▌ █ Show databases: ▐▌ █ Show databases; ▐▌ █ View scheduler settings: ▐▌ █ SELECT @@event_scheduler; ▐▌ █ Create new database, name would be test for example: ▐▌ █ create database test; ▐▌ █ Find HERE a list of some other common mysql cmds. ▐▌ █ Find HERE and HERE a fix myisam problems. ▐▌ █ ▐▌ █ After upgrading your mysql you might notice some processes running that ▐▌ █ weren't before depending on your setup. These are normal and the ▐▌ █ daemon.error refers to the logging facility of mysql: ▐▌ █ ps x or ps ax | grep mysql or ps -ef | grep -in mysql ▐▌ █ 31097 ? S 0:00 /bin/sh /usr/bin/mysqld_safe ▐▌ █ 31212 ? Sl 52:47 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/run/mysqld/mysqld.sock --port=3306 █ 31214 ? S 0:00 logger -t mysqld -p daemon.error ▐▌ █ ▐▌ █ //---------------------------------------------------------------------- ▐▌ █ ▐▌ █ If you find mistakes, have suggestions, and or questions please post at ▐▌ █ mewbies forum HERE - thank you. ▐▌ █ ▐▌ █ Last update on 25 Aug '10 ▐▌ █ ▐▌ █▌ █▌ █▌ - mewbies.com - █▌ █▌ █▌ ██▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄