Investigating Error Messages in Greenplum

Greenplum Database log messages are written to files in the pg_log directory within the master’s or segment’s data directory. Because the master log file contains the most information, you should always check it first. 

a) Log files roll over daily and use the naming convention: gpdb-YYYY-MM-DD_hhmmss.csv. 

b) To locate the log files on the master host:

$ cd $MASTER_DATA_DIRECTORY/pg_log

c) Log lines have the format of:

timestamp | user | database | statement_id | con# cmd# |:-LOG_LEVEL: log_message

Log files can be bulky so you may want to focus your search for WARNING, ERROR, FATAL or PANIC log level messages. You can use the Greenplum utility gplogfilter to search through Greenplum Database log files. For example, when you run the following command on the master host, it checks for problem log messages in the standard logging locations:

$ gplogfilter -t

To search for related log entries in the segment log files, you can run gplogfilter on the segment hosts using gpssh

You can identify corresponding log entries by the statement_id or con# (session identifier). 

For example, to search for log messages in the segment log files containing the string sachi and save output to a file:

gpssh -f seg_hosts_file -e 'source /usr/local/greenplum-db/greenplum_path.sh ; gplogfilter -f sachi /gpdata/*/pg_log/gpdb*.csv' > seglog.out