Graylog output plugin JDBC
Sends log from a steam to a traditional RDMBS database via JDBC.
How to build
Use mvn clean package
to create JAR
See Providing JDBC driver on how to adjust JAR for your RDBMS
Put prepared JAR inside Graylog plugins folder
Put a JDBC driver inside Graylog plugins folder
Restart Graylog
Create new output globally or inside stream
Connect output to your stream
Make sure your RDBMS commit speed is sufficient to handle output message rate
Providing JDBC driver
Graylog does not ship with JDBC drivers.
You must add your driver JAR manually to plugins folder and update graylog-output-jdbc-2.5.1.jar accordingly.
Add following line to graylog2-output-jdbc-2.5.1.jar/META-INF/MANIFEST.MF
Class-Path: driver.jar
Example:
Class-Path: mysql-connector-java-8.0.17.jar
How to update using command line?
Create my-manifest.mf
and use jar tool from JDK:
jar -uvmf my-manifest.mf target/graylog-output-jdbc-2.5.1.jar
Output options
Driver class to use (optional)
Driver class to initialize. Needed so URL can be handled properly. Recent JDK 8+ have autodiscovery of JDBC drivers so it might be not needed.
Default value: NONE
Example: com.mysql.jdbc.Driver
JDBC URL
Fully qualified JDBC url to connect to.
Default value: NONE
Example: jdbc:mysql://localhost:3307/graylog
Username (optional)
Username to connect to RDBMS as. If not specified, no password will be passed to driver.
Default value: NONE
Password (optional)
Password for user.
Default value: NONE
Additional fields (optional)
Comma separated list of additional attributes for Message insert query. I.e. if you have custom attribute defined in your log i.e. hostname
, specify it here and adjust a query accordingly, i.e. insert into log (message_date, message_id, source, message, hostname) values (?, ?, ?, ?, ?)
This way you don`t need to use log_attribute table, greatly increasing commit speed.
Default value: NONE
Message insert query
Query to execute to add log entry. Must contain required 4 columns and optional (see Additional fields). Must produce generated key (ID).
Default value: insert into log (message_date, message_id, source, message) values (?, ?, ?, ?)
Attribute insert query (optional)
If specified all attributes will be added using this query.
Default value: insert into log_attribute (log_id, name, value) values (?, ?, ?)
Table definition
By default output uses table 'log' for main message entry and 'log_attribute' for attributes.
Sample table creation script (MySQL):
create table if not exists log (
id int not null auto_increment,
message_date datetime,
message_id varchar (64 ),
source varchar (32 ),
message varchar (4096 ) null ,
PRIMARY KEY (id)
);
create table if not exists log_attribute (
id int not null auto_increment,
log_id numeric (10 ,0 ),
name varchar (255 ),
value varchar (4096 ) null ,
primary key (id),
foreign key (log_id) references log(id)
);
Make sure that for table log, column ID are generated automatically. For example add 'identity' (MS SQL/Sybase ASE) or AUTO_INCREMENT (MySQL) into column definition.
Links
Message from maintainer:
It would be great if you create an issue on GitHub source repo (https://github.com/wizecore/graylog2-output-jdbc) if experience any problems instead of commenting here.