Getting Started With Graylog2 for Logging

I have written a new post for installing the new version of Graylog2, 0.9.6. and sending logs to it - Check it out!

In this blog post I will show you how to install and setup Graylog2 to manage logs on a Ubuntu 11.04 64bit server.

I have had problems with loggly.com. Not the service itself but that rsyslog keeps not wanting to send logs all the time. Because of this I am going to setup graylog2 in a virtual machine to try it out.

The only application I installed when I created the virtual machine was OpenSSH server. I won’t go through the installation of Ubuntu Server because it’s simple enough.

The first step I did after installing my Ubuntu virtual machine, is setting the static IP to 192.168.1.150.

Here are the steps involved in getting Graylog2 setup on a clean Ubuntu Server. The easiest way to get it running is to use this tutorial.

Server Installation

As always, on a new installation, you should update sudo apt-key update sudo apt-get upgrade
Install OpenJDK and its dependencies. This can take a while, there is a lot to install sudo apt-get install openjdk-6-jre
Go and download the latest stable release of Graylog2. The version I used is 0.9.5p1 wget https://github.com/downloads/Graylog2/graylog2-server/graylog2-server-0.9.5p1.tar.gz
Extract the downloaded archive and change to it tar xvfz graylog2-server-0.9.5p1.tar.gz cd graylog2-server-0.9.5p1
Copy the example configuration to /etc/graylog2.conf sudo cp graylog2.conf.example /etc/graylog2.conf
You can leave all settings except the MongoDB login details and the messages collection size as they are for the moment.
Now we need to install MongoDB, the database server which Graylog2 uses. To do this run apt-key adv –keyserver keyserver.ubuntu.com –recv 7F0CEB10 This will ensure that the database software you download is actually the software published by 10gen.
Now we need to add the 10gen repository on the system. Do this by running sudo nano /etc/apt/sources.list and add deb http://downloads.mongodb.org/distros/ubuntu 10.4 10gen to the end.
Update the repository list and install MongoDB sudo apt-get update sudo apt-get install mongodb-stable
When MongoDB has finished installing, run mongo. You should see something similar to this. If so you have successfully installed MongoDB. Press Ctrl+D to exit the shell.

Mongo

Now we need to create an admin user for mongo and add a user to mongo for the ‘graylog2’ collection. mongo use admin db.addUser(‘admin’, ‘password’) db.auth(‘admin’, ‘password’) use graylog2 db.addUser(‘grayloguser’, ‘grayloguser-password’)
Now set mongodb_user and mongodb_password in /etc/graylog2.conf
Turn Mongo security on in /etc/mongodb.conf. By default it’s off. Find #auth = true and uncomment it so it looks like auth = true. Remove the # sign
Reboot the mongodb service service mongodb restart
Change back to the graylog2-server-0.9.5p1 folder and then into bin cd graylog2-server-0.9.5p1 && cd bin
Start the graylog2 server ./graylog2ctl start

Web Interface Installation

*+ Install the required packaged sudo apt-get install ruby1.8 rubygems rake make libopenssl-ruby ruby-dev build-essential git-core

Go and download the latest stable release of the Graylog2 web interface. The version I used is 0.9.5p2 wget https://github.com/downloads/Graylog2/graylog2-web-interface/graylog2-web-interface-0.9.5p2.tar.gz
Extract the downloaded archive tar xvfz graylog2-web-interface-0.9.5p2.tar.gz
Becuase the installation of RubyGems is old (mine was 1.3.5), it needs to be updated to be able to download the latest gems. cd ~ && wget http://production.cf.rubygems.org/rubygems/rubygems-1.8.10.tgz tar xvfz rubygems-1.8.10.tgz sudo ruby setup.rb
Install the Bundler gem sudo gem install bundler
Change to the graylog2 web interface folder & install required gems cd graylog2-web-interface-0.9.5p2 sudo bundle install
Edit all *.yml files in the config folder. They should be pretty self-explanatory and commented. The mongoid.yml has to be the same details as graylog2-server uses. nano config/email.yml nano config/general.yml nano config/mongoid.yml
If you don’t want the web interface to ask graylog2.org for version updates, disable them in config/general.yml
Run the server. This launches a web server on port 3000 script/rails server -e production
It is best to serve Graylog2 over apache instead of using WEBrick. We will be installing Apache2 and Passenger. Follow through the instructions. When it comes to editing the Apache configuration file, add the contents to /etc/apache/httpd.conf. sudo apt-get install apache2 libcurl4-openssl-dev apache2-prefork-dev libapr1-dev sudo apt-get install libcurl4-openssl-dev apache2-prefork-dev libapr1-dev sudo gem install passenger sudo passenger-install-apache2-module sudo /etc/init.d/apache2 restart
Copy the graylog2-web-interface-0.9.5p2 to /var/www/graylog2-web-interface-0.9 cp -R graylog2-web-interface-0.9.5p2 /var/www/graylog2-web-interface-0.9
Configure a virtual host sudo nano /etc/apache2/sites-available/graylog2
Copy this and paste it into the graylog2 file you just created. Change what’s relevant for your system. The common things to change is the IP, ServerName and ServerAlias. <VirtualHost 111.111.111.111:80> ServerAdmin [email protected] ServerName graylog2.example.com ServerAlias graylog2.example.com DocumentRoot /var/www/graylog2-web-interface-0.9/public
```
  	<Directory /var/www/graylog2-web-interface-0.9/public>
      	Allow from all
      	Options -MultiViews
  		</Directory>

  		ErrorLog /var/log/apache2/error.log
  	LogLevel warn
  	CustomLog /var/log/apache2/access.log combined
  </VirtualHost>
```
Enable the virtual host you have just created sudo a2ensite graylog2
Reload apache to activate the new configuration. It says to reload but I always restart. sudo /etc/init.d/apache2 restart
Now access graylog2.example.com and you should see the Graylog2 web interface which you saw earlier.
Congratulations. You have now setup a Graylog2 server

In another post I will go though how to add a host and setup log forwarding to the new Graylog2 server we have just setup.