Greenplum Database Log Files

GPDB Log Files


Know the location and content of system log files and monitor them on a regular basis and not just when problems arise.


PATH

DESCRIPTION

/var/gpadmin/gpadminlogs/*

Many different types of log files, directory on each server

/var/gpadmin/gpadminlogs/gpstart_ date.log

start log

/var/gpadmin/gpadminlogs/gpstop_ date.log

stop log

/var/gpadmin/gpadminlogs/gpsegstart.py_idb*gpadmin_ date.log

segment start log

/var/gpadmin/gpadminlogs/gpsegstop.py_idb*gpadmin_ date.log

segment stop log

/var/gpdb/ instance /datamaster/gpseg-1/pg_log/startup.log

instance start log

/var/gpdb/ instance /datamaster/gpseg-1/gpperfmon/logs/gpmon.*.log

gpperfmon logs

/var/gpdb/ instance /datamirror/gpseg n /pg_log/*.csv

mirror segment logs

/var/gpdb/ instance /dataprimary/gpseg n /pg_log/*.csv

primary segment logs

/var/log/messages

global linux system messages


Note: date is format YYYYMMDD, instance is your current instance name, n is a segment number


1. Greenplum Database Log Files

Greenplum Database log output tends to be voluminous (especially at higher debug levels) and you do not need to save it indefinitely. Administrators need to rotate the log files periodically so that new log files are started and old ones are removed after a reasonable period of time.

Greenplum Database has log file rotation enabled on the master and all segment instances. Daily log files are created in pg_log of the master and each segment data directory using the naming convention of: gpdb-YYYY-MM-DD.log. Although log files are rolled over daily, they are not automatically truncated or deleted. Administrators will need to implement some script or program to periodically clean up old log files in the pg_log directory of the master and each segment instance.

[09:44 sachi@sachi ~] > su - gpadmin
Password: 
[gpadmin@sachi ~]$ env
HOSTNAME=sachi
MASTER_DATA_DIRECTORY=/home/gpmaster/gpsne-1
SHELL=/bin/bash
TERM=xterm
HISTSIZE=1000
GPHOME=/usr/local/greenplum-db/.
QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
USER=gpadmin
LD_LIBRARY_PATH=/usr/local/greenplum-db/./lib:/usr/local/greenplum-db/./ext/python/lib:
MAIL=/var/spool/mail/gpadmin
PATH=/usr/local/greenplum-db/./bin:/usr/local/greenplum-db/./ext/python/bin:/usr/lib64/qt-.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/gpadmin/bin
PWD=/home/gpadmin
LANG=en_US.UTF-8
PYTHONHOME=/usr/local/greenplum-db/./ext/python
SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
HISTCONTROL=ignoredups
SHLVL=1
HOME=/home/gpadmin
OPENSSL_CONF=/usr/local/greenplum-db/./etc/openssl.cnf
PYTHONPATH=/usr/local/greenplum-db/./lib/python
LOGNAME=gpadmin
QTLIB=/usr/lib64/qt-3.3/lib
CVS_RSH=ssh
LESSOPEN=|/usr/bin/lesspipe.sh %s
DISPLAY=:0.0
G_BROKEN_FILENAMES=1
XAUTHORITY=/home/gpadmin/.xauthT5AvoM
_=/bin/env
[gpadmin@sachi ~]$ cd $MASTER_DATA_DIRECTORY
[gpadmin@sachi gpsne-1]$ ls -l
total 156
drwx------. 10 gpadmin gpadmin  4096 Aug  2 11:40 base
drwx------.  2 gpadmin gpadmin  4096 Oct 31 14:23 global
-r--------.  1 gpadmin gpadmin   109 Apr 23  2013 gp_dbid
drwxrwxr-x.  5 gpadmin gpadmin  4096 Apr 23  2013 gpperfmon
drwx------.  2 gpadmin gpadmin  4096 Apr 23  2013 pg_changetracking
drwx------.  2 gpadmin gpadmin  4096 Apr 23  2013 pg_clog
drwx------.  2 gpadmin gpadmin  4096 Nov 15 18:58 pg_distributedlog
drwx------.  2 gpadmin gpadmin  4096 Apr 23  2013 pg_distributedxidmap
-rw-rw-r--.  1 gpadmin gpadmin  4764 Oct 30 11:14 pg_hba.conf
-rw-rw-r--.  1 gpadmin gpadmin  4346 Apr 24  2013 pg_hba.conf.orig
-rw-------.  1 gpadmin gpadmin  1636 Apr 23  2013 pg_ident.conf
drwx------.  3 gpadmin gpadmin  4096 Nov 17 00:00 pg_log
drwx------.  4 gpadmin gpadmin  4096 Apr 23  2013 pg_multixact
drwx------.  2 gpadmin gpadmin  4096 Oct 31 14:23 pg_stat_tmp
drwx------.  2 gpadmin gpadmin  4096 Nov  8 02:53 pg_subtrans
drwx------.  2 gpadmin gpadmin  4096 Apr 23  2013 pg_tblspc
drwx------.  2 gpadmin gpadmin  4096 Apr 23  2013 pg_twophase
drwx------.  2 gpadmin gpadmin  4096 Oct  9 08:44 pg_utilitymodedtmredo
-rw-------.  1 gpadmin gpadmin     4 Apr 23  2013 PG_VERSION
drwx------.  3 gpadmin gpadmin  4096 Nov 15 12:48 pg_xlog
-rw-------.  1 gpadmin gpadmin 19364 Aug 31 19:20 postgresql.conf
-rw-------.  1 gpadmin gpadmin 19336 Aug 31 19:20 postgresql.conf.bak
-rw-------.  1 gpadmin gpadmin 19299 Apr 28  2013 postgresql.conf.orig
-rw-------.  1 gpadmin gpadmin   169 Oct 31 14:23 postmaster.opts
-rw-------.  1 gpadmin gpadmin    48 Oct 31 14:23 postmaster.pid
[gpadmin@sachi gpsne-1]$ cd pg_log
[gpadmin@sachi pg_log]$ ls -l
total 854200
drwxrwxr-x. 2 gpadmin gpadmin     4096 Aug  7 13:14 backuplog
-rw-------. 1 gpadmin gpadmin     6087 Aug  7 13:21 gpdb-2013-08-07_132121.csv
-rw-------. 1 gpadmin gpadmin     9031 Aug  8 09:50 gpdb-2013-08-08_000000.csv
-rw-------. 1 gpadmin gpadmin     2205 Aug  9 08:46 gpdb-2013-08-09_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 10 00:00 gpdb-2013-08-10_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 11 00:00 gpdb-2013-08-11_000000.csv
-rw-------. 1 gpadmin gpadmin    75197 Aug 12 11:53 gpdb-2013-08-12_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 13 00:00 gpdb-2013-08-13_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 14 00:00 gpdb-2013-08-14_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 15 00:00 gpdb-2013-08-15_000000.csv
-rw-------. 1 gpadmin gpadmin     4118 Aug 16 18:58 gpdb-2013-08-16_000000.csv
-rw-------. 1 gpadmin gpadmin    32591 Aug 19 09:10 gpdb-2013-08-19_091024.csv
-rw-------. 1 gpadmin gpadmin   192871 Aug 19 22:05 gpdb-2013-08-19_091034.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 20 00:00 gpdb-2013-08-20_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 21 00:00 gpdb-2013-08-21_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 22 00:00 gpdb-2013-08-22_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 23 00:00 gpdb-2013-08-23_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 24 00:00 gpdb-2013-08-24_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 25 00:00 gpdb-2013-08-25_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 26 00:00 gpdb-2013-08-26_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 27 00:00 gpdb-2013-08-27_000000.csv
-rw-------. 1 gpadmin gpadmin        0 Aug 28 00:00 gpdb-2013-08-28_000000.csv
-rw-------. 1 gpadmin gpadmin   222572 Aug 29 14:27 gpdb-2013-08-29_000000.csv
-rw-------. 1 gpadmin gpadmin   163875 Aug 30 16:23 gpdb-2013-08-30_000000.csv
-rw-------. 1 gpadmin gpadmin   171626 Aug 31 19:23 gpdb-2013-08-31_000000.csv
-rw-------. 1 gpadmin gpadmin    32342 Aug 31 19:27 gpdb-2013-08-31_192712.csv
-rw-------. 1 gpadmin gpadmin  2731420 Aug 31 23:59 gpdb-2013-08-31_192722.csv
-rw-------. 1 gpadmin gpadmin  8763728 Sep  1 14:43 gpdb-2013-09-01_000000.csv
-rw-------. 1 gpadmin gpadmin    32152 Sep  1 14:44 gpdb-2013-09-01_144357.csv
-rw-------. 1 gpadmin gpadmin  5515357 Sep  1 23:58 gpdb-2013-09-01_144403.csv
-rw-------. 1 gpadmin gpadmin 14270054 Sep  2 23:59 gpdb-2013-09-02_000000.csv
-rw-------. 1 gpadmin gpadmin 14272263 Sep  3 23:59 gpdb-2013-09-03_000000.csv
-rw-------. 1 gpadmin gpadmin 14275574 Sep  4 23:58 gpdb-2013-09-04_000000.csv
-rw-------. 1 gpadmin gpadmin 14278558 Sep  5 23:58 gpdb-2013-09-05_000000.csv
-rw-------. 1 gpadmin gpadmin 14296858 Sep  6 23:59 gpdb-2013-09-06_000000.csv
-rw-------. 1 gpadmin gpadmin 14296439 Sep  7 23:59 gpdb-2013-09-07_000000.csv
-rw-------. 1 gpadmin gpadmin 14294899 Sep  8 23:58 gpdb-2013-09-08_000000.csv
-rw-------. 1 gpadmin gpadmin 14315211 Sep 10 00:00 gpdb-2013-09-09_000000.csv
-rw-------. 1 gpadmin gpadmin 14296641 Sep 10 23:59 gpdb-2013-09-10_000000.csv
-rw-------. 1 gpadmin gpadmin 14307304 Sep 11 23:58 gpdb-2013-09-11_000000.csv
-rw-------. 1 gpadmin gpadmin 14319568 Sep 12 23:58 gpdb-2013-09-12_000000.csv
-rw-------. 1 gpadmin gpadmin 14281322 Sep 13 23:59 gpdb-2013-09-13_000000.csv
-rw-------. 1 gpadmin gpadmin 14323672 Sep 14 23:59 gpdb-2013-09-14_000000.csv
-rw-------. 1 gpadmin gpadmin 14321730 Sep 15 23:58 gpdb-2013-09-15_000000.csv
-rw-------. 1 gpadmin gpadmin 14323806 Sep 16 23:58 gpdb-2013-09-16_000000.csv
-rw-------. 1 gpadmin gpadmin 14322834 Sep 17 23:58 gpdb-2013-09-17_000000.csv
-rw-------. 1 gpadmin gpadmin 14329812 Sep 18 23:59 gpdb-2013-09-18_000000.csv
-rw-------. 1 gpadmin gpadmin 14281094 Sep 19 23:58 gpdb-2013-09-19_000000.csv
-rw-------. 1 gpadmin gpadmin 14321798 Sep 20 23:58 gpdb-2013-09-20_000000.csv
-rw-------. 1 gpadmin gpadmin  7734394 Sep 21 12:54 gpdb-2013-09-21_000000.csv
-rw-------. 1 gpadmin gpadmin    33612 Oct  9 08:44 gpdb-2013-10-09_084401.csv
-rw-------. 1 gpadmin gpadmin    32716 Oct  9 08:44 gpdb-2013-10-09_084405.csv
-rw-------. 1 gpadmin gpadmin  9138791 Oct  9 23:59 gpdb-2013-10-09_084414.csv
-rw-------. 1 gpadmin gpadmin 14246896 Oct 10 23:59 gpdb-2013-10-10_000000.csv
-rw-------. 1 gpadmin gpadmin 14255490 Oct 11 23:58 gpdb-2013-10-11_000000.csv
-rw-------. 1 gpadmin gpadmin 14277191 Oct 12 23:59 gpdb-2013-10-12_000000.csv
-rw-------. 1 gpadmin gpadmin 14256064 Oct 13 23:59 gpdb-2013-10-13_000000.csv
-rw-------. 1 gpadmin gpadmin 14255062 Oct 14 23:58 gpdb-2013-10-14_000000.csv
-rw-------. 1 gpadmin gpadmin 14256818 Oct 15 23:58 gpdb-2013-10-15_000000.csv
-rw-------. 1 gpadmin gpadmin 14276670 Oct 16 23:59 gpdb-2013-10-16_000000.csv
-rw-------. 1 gpadmin gpadmin 14255144 Oct 17 23:59 gpdb-2013-10-17_000000.csv
-rw-------. 1 gpadmin gpadmin 14254751 Oct 18 23:58 gpdb-2013-10-18_000000.csv
-rw-------. 1 gpadmin gpadmin 14293302 Oct 19 23:59 gpdb-2013-10-19_000000.csv
-rw-------. 1 gpadmin gpadmin 14284054 Oct 20 23:59 gpdb-2013-10-20_000000.csv
-rw-------. 1 gpadmin gpadmin 14282041 Oct 21 23:58 gpdb-2013-10-21_000000.csv
-rw-------. 1 gpadmin gpadmin 14282362 Oct 22 23:58 gpdb-2013-10-22_000000.csv
-rw-------. 1 gpadmin gpadmin 14301951 Oct 23 23:59 gpdb-2013-10-23_000000.csv
-rw-------. 1 gpadmin gpadmin 14281774 Oct 24 23:59 gpdb-2013-10-24_000000.csv
-rw-------. 1 gpadmin gpadmin 14280320 Oct 25 23:58 gpdb-2013-10-25_000000.csv
-rw-------. 1 gpadmin gpadmin 14269319 Oct 26 23:58 gpdb-2013-10-26_000000.csv
-rw-------. 1 gpadmin gpadmin 14300974 Oct 27 23:59 gpdb-2013-10-27_000000.csv
-rw-------. 1 gpadmin gpadmin 14281638 Oct 28 23:59 gpdb-2013-10-28_000000.csv
-rw-------. 1 gpadmin gpadmin 14290652 Oct 29 23:58 gpdb-2013-10-29_000000.csv
-rw-------. 1 gpadmin gpadmin  6642334 Oct 30 11:06 gpdb-2013-10-30_000000.csv
-rw-------. 1 gpadmin gpadmin      375 Oct 30 11:06 gpdb-2013-10-30_110618.csv
-rw-------. 1 gpadmin gpadmin      375 Oct 30 11:07 gpdb-2013-10-30_110722.csv
-rw-------. 1 gpadmin gpadmin      379 Oct 30 11:08 gpdb-2013-10-30_110854.csv
-rw-------. 1 gpadmin gpadmin      377 Oct 30 11:09 gpdb-2013-10-30_110935.csv
-rw-------. 1 gpadmin gpadmin      379 Oct 30 11:10 gpdb-2013-10-30_111007.csv
-rw-------. 1 gpadmin gpadmin    32335 Oct 30 11:10 gpdb-2013-10-30_111054.csv
-rw-------. 1 gpadmin gpadmin    49662 Oct 30 11:11 gpdb-2013-10-30_111103.csv
-rw-------. 1 gpadmin gpadmin    32335 Oct 30 11:11 gpdb-2013-10-30_111144.csv
-rw-------. 1 gpadmin gpadmin    50168 Oct 30 11:13 gpdb-2013-10-30_111149.csv
-rw-------. 1 gpadmin gpadmin      371 Oct 30 11:13 gpdb-2013-10-30_111339.csv
-rw-------. 1 gpadmin gpadmin    32335 Oct 30 11:14 gpdb-2013-10-30_111414.csv
-rw-------. 1 gpadmin gpadmin  8558669 Oct 30 23:58 gpdb-2013-10-30_111422.csv
-rw-------. 1 gpadmin gpadmin  9307879 Oct 31 14:10 gpdb-2013-10-31_000000.csv
-rw-------. 1 gpadmin gpadmin    32546 Oct 31 14:23 gpdb-2013-10-31_142310.csv
-rw-------. 1 gpadmin gpadmin  6418018 Oct 31 23:58 gpdb-2013-10-31_142320.csv
-rw-------. 1 gpadmin gpadmin 15239313 Nov  1 23:59 gpdb-2013-11-01_000000.csv
-rw-------. 1 gpadmin gpadmin 15238170 Nov  2 23:59 gpdb-2013-11-02_000000.csv
-rw-------. 1 gpadmin gpadmin 15239228 Nov  3 22:59 gpdb-2013-11-03_000000.csv
-rw-------. 1 gpadmin gpadmin   615663 Nov  3 23:59 gpdb-2013-11-03_230000.csv
-rw-------. 1 gpadmin gpadmin 15220615 Nov  4 23:59 gpdb-2013-11-04_000000.csv
-rw-------. 1 gpadmin gpadmin 15238722 Nov  5 23:59 gpdb-2013-11-05_000000.csv
-rw-------. 1 gpadmin gpadmin 15237171 Nov  6 23:58 gpdb-2013-11-06_000000.csv
-rw-------. 1 gpadmin gpadmin 15237929 Nov  7 23:58 gpdb-2013-11-07_000000.csv
-rw-------. 1 gpadmin gpadmin 15239387 Nov  8 23:58 gpdb-2013-11-08_000000.csv
-rw-------. 1 gpadmin gpadmin 15238297 Nov  9 23:58 gpdb-2013-11-09_000000.csv
-rw-------. 1 gpadmin gpadmin 15282922 Nov 10 23:59 gpdb-2013-11-10_000000.csv
-rw-------. 1 gpadmin gpadmin 15271750 Nov 11 23:59 gpdb-2013-11-11_000000.csv
-rw-------. 1 gpadmin gpadmin 15413144 Nov 12 23:58 gpdb-2013-11-12_000000.csv
-rw-------. 1 gpadmin gpadmin 15281677 Nov 13 23:59 gpdb-2013-11-13_000000.csv
-rw-------. 1 gpadmin gpadmin 15288399 Nov 14 23:59 gpdb-2013-11-14_000000.csv
-rw-------. 1 gpadmin gpadmin 15404663 Nov 15 23:59 gpdb-2013-11-15_000000.csv
-rw-------. 1 gpadmin gpadmin 15285652 Nov 16 23:59 gpdb-2013-11-16_000000.csv
-rw-------. 1 gpadmin gpadmin  5978173 Nov 17 09:44 gpdb-2013-11-17_000000.csv
-r--------. 1 gpadmin gpadmin      105 Oct 31 14:23 gp_era
-rw-------. 1 gpadmin gpadmin    27526 Oct 31 14:23 startup.log
[gpadmin@sachi pg_log]$ 

[gpadmin@sachi pg_log]$ tail -10 gpdb-2013-11-17_000000.csv
2013-11-17 09:48:06.340148 EST,"gpmon","gpperfmon",p28064,th745953024,"[local]",,2013-11-17 09:48:06 EST,0,con175656,cmd2,seg-1,,,,,"LOG","00000","disconnection: session time: 0:00:00.115 user=gpmon database=gpperfmon host=[local]",,,,,,,0,,"postgres.c",5432,
2013-11-17 09:48:06.340253 EST,"gpmon","gpperfmon",p28072,th745953024,"[local]",,2013-11-17 09:48:06 EST,0,con175656,,seg-1,,,,,"LOG","00000","disconnection: session time: 0:00:00.097 user=gpmon database=gpperfmon host=[local]",,,,,,,0,,"postgres.c",5432,
2013-11-17 09:48:06.340903 EST,,,p28080,th745953024,,,2013-11-17 09:48:06 EST,0,,,seg-1,,,,,"LOG","00000","connection received: host=[local]",,,,,,,0,,"postmaster.c",6580,
2013-11-17 09:48:06.341233 EST,"gpmon","gpperfmon",p28080,th745953024,"[local]",,2013-11-17 09:48:06 EST,0,,,seg-1,,,,,"LOG","00000","connection authorized: user=gpmon database=gpperfmon",,,,,,,0,,"postmaster.c",6666,
2013-11-17 09:48:06.344216 EST,"gpmon","gpperfmon",p28080,th745953024,"[local]",,2013-11-17 09:48:06 EST,944593,con175658,cmd1,seg-1,,dx111499,x944593,sx1,"LOG","00000","statement: insert into diskspace_history select * from _diskspace_tail;",,,,,,"insert into diskspace_history select * from _diskspace_tail;",0,,"postgres.c",1542,
2013-11-17 09:48:06.357041 EST,,,p28088,th745953024,,,2013-11-17 09:48:06 EST,0,,,seg-1,,,,,"LOG","00000","connection received: host=[local]",,,,,,,0,,"postmaster.c",6580,
2013-11-17 09:48:06.357593 EST,"gpmon","gpperfmon",p28088,th745953024,"[local]",,2013-11-17 09:48:06 EST,0,con175658,,seg-1,,,,,"LOG","00000","connection authorized: user=gpmon database=gpperfmon",,,,,,,0,,"postmaster.c",6666,
2013-11-17 09:48:06.361118 EST,"gpmon","gpperfmon",p28088,th745953024,"[local]",,2013-11-17 09:48:06 EST,944593,con175658,cmd2,seg-1,slice1,dx111499,x944593,sx1,"LOG","00000","statement: insert into diskspace_history select * from _diskspace_tail;",,,,,,"insert into diskspace_history select * from _diskspace_tail;",0,,"postgres.c",1093,
2013-11-17 09:48:06.455713 EST,"gpmon","gpperfmon",p28080,th745953024,"[local]",,2013-11-17 09:48:06 EST,0,con175658,cmd2,seg-1,,,,,"LOG","00000","disconnection: session time: 0:00:00.114 user=gpmon database=gpperfmon host=[local]",,,,,,,0,,"postgres.c",5432,
2013-11-17 09:48:06.455781 EST,"gpmon","gpperfmon",p28088,th745953024,"[local]",,2013-11-17 09:48:06 EST,0,con175658,,seg-1,,,,,"LOG","00000","disconnection: session time: 0:00:00.098 user=gpmon database=gpperfmon host=[local]",,,,,,,0,,"postgres.c",5432,

[gpadmin@sachi pg_log]$ cat startup.log
2013-04-23 19:55:19.568524 EDT,,,p9481,th857798400,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-23 19:55:19.568735 EDT,,,p9481,th857798400,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-23 19:58:01.772528 EDT,,,p17082,th1321957120,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-23 19:58:01.772775 EDT,,,p17082,th1321957120,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-23 19:58:07.103638 EDT,,,p17520,th1441867520,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-23 19:58:07.103873 EDT,,,p17520,th1441867520,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-24 17:11:10.015426 EDT,,,p4423,th-57956608,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-24 17:11:10.031787 EDT,,,p4423,th-57956608,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-24 17:11:22.950310 EDT,,,p4865,th-1123461376,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-24 17:11:22.950557 EDT,,,p4865,th-1123461376,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-26 20:51:26.625471 EDT,,,p3587,th-1646864640,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-26 20:51:26.641780 EDT,,,p3587,th-1646864640,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-26 20:52:10.006366 EDT,,,p3648,th-183769344,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-26 20:52:10.006602 EDT,,,p3648,th-183769344,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-26 20:53:43.432004 EDT,,,p3810,th-876996864,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-26 20:53:43.432232 EDT,,,p3810,th-876996864,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-26 20:53:58.602959 EDT,,,p4252,th-936077568,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-26 20:53:58.603182 EDT,,,p4252,th-936077568,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-26 22:34:50.492042 EDT,,,p4454,th-144845056,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-26 22:34:50.508308 EDT,,,p4454,th-144845056,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-26 22:34:53.842494 EDT,,,p4489,th-214759680,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-26 22:34:53.842692 EDT,,,p4489,th-214759680,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-26 22:35:05.277285 EDT,,,p4930,th312977152,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-26 22:35:05.277488 EDT,,,p4930,th312977152,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-28 19:55:10.733857 EDT,,,p8089,th-1641515264,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-28 19:55:10.734334 EDT,,,p8089,th-1641515264,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-04-28 19:55:18.964296 EDT,,,p8527,th-15378688,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-04-28 19:55:18.964554 EDT,,,p8527,th-15378688,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-05-13 08:04:29.271637 EDT,,,p9076,th-1415391488,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-05-13 08:04:29.272522 EDT,,,p9076,th-1415391488,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-05-13 08:04:39.693617 EDT,,,p9520,th971040512,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-05-13 08:04:39.693856 EDT,,,p9520,th971040512,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-05-25 16:24:30.306097 EDT,,,p5561,th-1264290048,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-05-25 16:24:30.362578 EDT,,,p5561,th-1264290048,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-05-25 16:24:52.480338 EDT,,,p6020,th1387489024,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-05-25 16:24:52.480586 EDT,,,p6020,th1387489024,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-05-31 15:12:46.721437 EDT,,,p17666,th72791808,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-05-31 15:12:46.722284 EDT,,,p17666,th72791808,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-05-31 15:12:50.155624 EDT,,,p17701,th2086233856,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-05-31 15:12:50.155863 EDT,,,p17701,th2086233856,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-05-31 15:13:01.288347 EDT,,,p18143,th-1922615552,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-05-31 15:13:01.288557 EDT,,,p18143,th-1922615552,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-06-10 09:04:47.873275 EDT,,,p797,th1528518400,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-06-10 09:04:48.384719 EDT,,,p797,th1528518400,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-06-10 09:05:01.885490 EDT,,,p1253,th-1668598016,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-06-10 09:05:01.885731 EDT,,,p1253,th-1668598016,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-07-24 09:00:10.397560 EDT,,,p6405,th-133245184,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-07-24 09:00:10.459549 EDT,,,p6405,th-133245184,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-07-24 09:00:16.863777 EDT,,,p6459,th-1202526464,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-07-24 09:00:16.864034 EDT,,,p6459,th-1202526464,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-07-24 09:02:01.699202 EDT,,,p6561,th-1093273856,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-07-24 09:02:01.699438 EDT,,,p6561,th-1093273856,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-07-24 09:06:02.627950 EDT,,,p7775,th1753749248,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-07-24 09:06:02.628203 EDT,,,p7775,th1753749248,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-07-24 13:07:45.776605 EDT,,,p15141,th1706170112,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-07-24 13:07:45.776853 EDT,,,p15141,th1706170112,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-07-24 13:18:50.793341 EDT,,,p15728,th1442494208,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-07-24 13:18:50.793590 EDT,,,p15728,th1442494208,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-07-24 13:21:50.512352 EDT,,,p15982,th-2143693056,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-07-24 13:21:50.512621 EDT,,,p15982,th-2143693056,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-07-24 13:27:07.217776 EDT,,,p16340,th-1615755520,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-07-24 13:27:07.218009 EDT,,,p16340,th-1615755520,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-07-24 13:27:17.630407 EDT,,,p16785,th-811636992,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-07-24 13:27:17.630647 EDT,,,p16785,th-811636992,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-07-25 14:50:02.848241 EDT,,,p22332,th562423552,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-07-25 14:50:02.848500 EDT,,,p22332,th562423552,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-07-25 14:50:08.277022 EDT,,,p22772,th1816745728,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-07-25 14:50:08.277263 EDT,,,p22772,th1816745728,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-01 08:50:31.800210 EDT,,,p7018,th453990144,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-01 08:50:31.800448 EDT,,,p7018,th453990144,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-01 08:50:37.253186 EDT,,,p7461,th128968448,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-01 08:50:37.253431 EDT,,,p7461,th128968448,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-02 11:37:27.233638 EDT,,,p8026,th-1823762688,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-02 11:37:27.251105 EDT,,,p8026,th-1823762688,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-02 11:37:37.115456 EDT,,,p8470,th18024192,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-02 11:37:37.115695 EDT,,,p8470,th18024192,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-03 14:57:08.253762 EDT,,,p32101,th-2133555456,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-03 14:57:08.262429 EDT,,,p32101,th-2133555456,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-03 14:57:15.580779 EDT,,,p32544,th-1437260032,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-03 14:57:15.581030 EDT,,,p32544,th-1437260032,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-03 15:00:49.337761 EDT,,,p32742,th1460745984,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-03 15:00:49.338003 EDT,,,p32742,th1460745984,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-03 15:00:54.641912 EDT,,,p730,th-1037363456,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-03 15:00:54.642160 EDT,,,p730,th-1037363456,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-07 13:20:56.006173 EDT,,,p3560,th-372832512,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-07 13:20:56.034025 EDT,,,p3560,th-372832512,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-07 13:21:20.934528 EDT,,,p4011,th859412224,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-07 13:21:20.934789 EDT,,,p4011,th859412224,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-19 09:10:24.156828 EDT,,,p30049,th-2143127808,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-19 09:10:24.173614 EDT,,,p30049,th-2143127808,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-19 09:10:33.991673 EDT,,,p30490,th388536064,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-19 09:10:33.991924 EDT,,,p30490,th388536064,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-31 19:27:12.536182 EDT,,,p3368,th759383808,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-31 19:27:12.548564 EDT,,,p3368,th759383808,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-08-31 19:27:22.183584 EDT,,,p3819,th-482359552,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-08-31 19:27:22.183821 EDT,,,p3819,th-482359552,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-09-01 14:43:57.935552 EDT,,,p27390,th319207168,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-09-01 14:43:57.935799 EDT,,,p27390,th319207168,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-09-01 14:44:03.281916 EDT,,,p27831,th-640006400,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-09-01 14:44:03.282152 EDT,,,p27831,th-640006400,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-09 08:44:01.600716 EDT,,,p6761,th-100927744,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-09 08:44:01.620095 EDT,,,p6761,th-100927744,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-09 08:44:05.105662 EDT,,,p6805,th-970160384,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-09 08:44:05.105895 EDT,,,p6805,th-970160384,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-09 08:44:14.734413 EDT,,,p7254,th706541312,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-09 08:44:14.734645 EDT,,,p7254,th706541312,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-30 11:06:18.777410 EDT,,,p25438,th109774592,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-30 11:06:18.777651 EDT,,,p25438,th109774592,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-30 11:07:22.647135 EDT,,,p25460,th513582848,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-30 11:07:22.647374 EDT,,,p25460,th513582848,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-30 11:08:54.153686 EDT,,,p25482,th-1856948480,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-30 11:08:54.153921 EDT,,,p25482,th-1856948480,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-30 11:09:35.025391 EDT,,,p25509,th-699083008,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-30 11:09:35.025628 EDT,,,p25509,th-699083008,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-30 11:10:07.040082 EDT,,,p25619,th-1737988352,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-30 11:10:07.040331 EDT,,,p25619,th-1737988352,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-30 11:10:54.504291 EDT,,,p25637,th-855595264,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-30 11:10:54.504522 EDT,,,p25637,th-855595264,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-30 11:11:03.844743 EDT,,,p26084,th-1921161472,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-30 11:11:03.844989 EDT,,,p26084,th-1921161472,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-30 11:11:44.535495 EDT,,,p26254,th2043639552,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-30 11:11:44.535735 EDT,,,p26254,th2043639552,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-30 11:11:49.886083 EDT,,,p26699,th-281606400,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-30 11:11:49.886324 EDT,,,p26699,th-281606400,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-30 11:13:39.518533 EDT,,,p26914,th2113234688,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-30 11:13:39.518773 EDT,,,p26914,th2113234688,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-30 11:14:14.308588 EDT,,,p26944,th-420722944,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-30 11:14:14.308819 EDT,,,p26944,th-420722944,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-30 11:14:21.943267 EDT,,,p27387,th-1393838336,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-30 11:14:21.943498 EDT,,,p27387,th-1393838336,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-31 14:23:10.546239 EDT,,,p4861,th723060480,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-31 14:23:10.547467 EDT,,,p4861,th723060480,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
2013-10-31 14:23:20.645638 EDT,,,p5307,th745953024,,,,0,,,seg-1,,,,,"LOG","00000","temporary files using default filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2501,
2013-10-31 14:23:20.645880 EDT,,,p5307,th745953024,,,,0,,,seg-1,,,,,"LOG","00000","transaction files using default pg_system filespace",,,,,,,,"primaryMirrorPopulateFilespaceInfo","primary_mirror_mode.c",2561,
[gpadmin@sachi pg_log]$ 


2. Management Utility Log Files

[gpadmin@sachi ~]$ pwd
/home/gpadmin
[gpadmin@sachi ~]$ ls -l
total 107512
drwxrwxr-x. 2 gpadmin gpadmin 4096 Oct 31 14:23 gpAdminLogs
-rw-rw-r--. 1 gpadmin gpadmin 4515 Jul 24 09:04 gpinitsystem_singlenode
-rwxr-xr-x. 1 root root 55684023 Oct 17 2012 greenplum-db-4.2.2.4-build-1-CE-RHEL5-x86_64.bin
-rw-r--r--. 1 root root 54377435 Apr 23 2013 greenplum-db-4.2.2.4-build-1-CE-RHEL5-x86_64.zip
-rwxr-xr-x. 1 gpadmin gpadmin 6 Apr 23 2013 hostlist_singlenode
-r--r--r--. 1 root root 7008 Oct 17 2012 README_INSTALL
drwxrwxr-x. 2 gpadmin gpadmin 4096 Apr 28 2013 sachi

Log files for the Greenplum Database management utilities are written to ~/gpAdminLogs by default. The naming convention for management log files is:
<script_name>_<date>.log .  The log entry format is:
<timestamp>:<utility>:<host>:<user>:[INFO|WARN|FATAL]:<message>
The log file for a particular utility execution is appended to its daily log file each time that utility is run.

[gpadmin@sachi ~]$ cd gpAdminLogs
[gpadmin@sachi gpAdminLogs]$ ls -l
total 692
-rw-rw-r--. 1 gpadmin gpadmin    48 Apr 23  2013 backout_gpinitsystem_gpadmin_20130423_195354
-rw-rw-r--. 1 gpadmin gpadmin   237 Aug 31 19:20 gpconfig_20130831.log
-rw-rw-r--. 1 gpadmin gpadmin 33174 Apr 24  2013 gpgetstatususingtransition.py_sachi:gpadmin_20130424.log
-rw-rw-r--. 1 gpadmin gpadmin   330 Apr 23  2013 gphostcachelookup.py_20130423.log
-rw-rw-r--. 1 gpadmin gpadmin 57169 Apr 23  2013 gpinitsystem_20130423.log
-rw-rw-r--. 1 gpadmin gpadmin  4144 Apr 24  2013 gpinitsystem_20130424.log
-rw-rw-r--. 1 gpadmin gpadmin  4144 Jul 24 09:04 gpinitsystem_20130724.log
-rw-rw-r--. 1 gpadmin gpadmin  2599 Aug 31 19:20 gpperfmon_install_20130831.log
-rw-rw-r--. 1 gpadmin gpadmin  3411 Apr 23  2013 gpsegstart.py_sachi:gpadmin_20130423.log
-rw-rw-r--. 1 gpadmin gpadmin  3405 Apr 24  2013 gpsegstart.py_sachi:gpadmin_20130424.log
-rw-rw-r--. 1 gpadmin gpadmin  7568 Apr 26  2013 gpsegstart.py_sachi:gpadmin_20130426.log
-rw-rw-r--. 1 gpadmin gpadmin  3405 Apr 28  2013 gpsegstart.py_sachi:gpadmin_20130428.log
-rw-rw-r--. 1 gpadmin gpadmin  3405 May 13  2013 gpsegstart.py_sachi:gpadmin_20130513.log
-rw-rw-r--. 1 gpadmin gpadmin  3405 May 25 16:24 gpsegstart.py_sachi:gpadmin_20130525.log
-rw-rw-r--. 1 gpadmin gpadmin  4169 May 31 15:13 gpsegstart.py_sachi:gpadmin_20130531.log
-rw-rw-r--. 1 gpadmin gpadmin  3405 Jun 10 09:05 gpsegstart.py_sachi:gpadmin_20130610.log
-rw-rw-r--. 1 gpadmin gpadmin  4169 Jul 24 13:27 gpsegstart.py_sachi:gpadmin_20130724.log
-rw-rw-r--. 1 gpadmin gpadmin  3411 Jul 25 14:50 gpsegstart.py_sachi:gpadmin_20130725.log
-rw-rw-r--. 1 gpadmin gpadmin  3405 Aug  1 08:50 gpsegstart.py_sachi:gpadmin_20130801.log
-rw-rw-r--. 1 gpadmin gpadmin  3405 Aug  2 11:37 gpsegstart.py_sachi:gpadmin_20130802.log
-rw-rw-r--. 1 gpadmin gpadmin  6810 Aug  3 15:00 gpsegstart.py_sachi:gpadmin_20130803.log
-rw-rw-r--. 1 gpadmin gpadmin  3405 Aug  7 13:21 gpsegstart.py_sachi:gpadmin_20130807.log
-rw-rw-r--. 1 gpadmin gpadmin  3411 Aug 19 09:10 gpsegstart.py_sachi:gpadmin_20130819.log
-rw-rw-r--. 1 gpadmin gpadmin  3405 Aug 31 19:27 gpsegstart.py_sachi:gpadmin_20130831.log
-rw-rw-r--. 1 gpadmin gpadmin  3411 Sep  1 14:44 gpsegstart.py_sachi:gpadmin_20130901.log
-rw-rw-r--. 1 gpadmin gpadmin  4163 Oct  9 08:44 gpsegstart.py_sachi:gpadmin_20131009.log
-rw-rw-r--. 1 gpadmin gpadmin 10233 Oct 30 11:14 gpsegstart.py_sachi:gpadmin_20131030.log
-rw-rw-r--. 1 gpadmin gpadmin  3405 Oct 31 14:23 gpsegstart.py_sachi:gpadmin_20131031.log
-rw-rw-r--. 1 gpadmin gpadmin   615 Apr 28  2013 gpsegstop.py_sachi:gpadmin_20130428.log
-rw-rw-r--. 1 gpadmin gpadmin   615 Jul 25 14:50 gpsegstop.py_sachi:gpadmin_20130725.log
-rw-rw-r--. 1 gpadmin gpadmin   615 Aug  1 08:50 gpsegstop.py_sachi:gpadmin_20130801.log
-rw-rw-r--. 1 gpadmin gpadmin  1230 Aug  3 15:00 gpsegstop.py_sachi:gpadmin_20130803.log
-rw-rw-r--. 1 gpadmin gpadmin   615 Aug  7 13:20 gpsegstop.py_sachi:gpadmin_20130807.log
-rw-rw-r--. 1 gpadmin gpadmin   615 Sep  1 14:43 gpsegstop.py_sachi:gpadmin_20130901.log
-rw-rw-r--. 1 gpadmin gpadmin  1845 Oct 30 11:13 gpsegstop.py_sachi:gpadmin_20131030.log
-rw-rw-r--. 1 gpadmin gpadmin  4110 Apr 23  2013 gpstart_20130423.log
-rw-rw-r--. 1 gpadmin gpadmin  5362 Apr 24  2013 gpstart_20130424.log
-rw-rw-r--. 1 gpadmin gpadmin 22451 Apr 26  2013 gpstart_20130426.log
-rw-rw-r--. 1 gpadmin gpadmin  5362 Apr 28  2013 gpstart_20130428.log
-rw-rw-r--. 1 gpadmin gpadmin  5362 May 13  2013 gpstart_20130513.log
-rw-rw-r--. 1 gpadmin gpadmin  5362 May 25 16:24 gpstart_20130525.log
-rw-rw-r--. 1 gpadmin gpadmin  6367 May 31 15:13 gpstart_20130531.log
-rw-rw-r--. 1 gpadmin gpadmin  5362 Jun 10 09:05 gpstart_20130610.log
-rw-rw-r--. 1 gpadmin gpadmin 38389 Jul 24 13:27 gpstart_20130724.log
-rw-rw-r--. 1 gpadmin gpadmin  4997 Jul 25 14:50 gpstart_20130725.log
-rw-rw-r--. 1 gpadmin gpadmin  4110 Aug  1 08:50 gpstart_20130801.log
-rw-rw-r--. 1 gpadmin gpadmin  5362 Aug  2 11:37 gpstart_20130802.log
-rw-rw-r--. 1 gpadmin gpadmin  8220 Aug  3 15:00 gpstart_20130803.log
-rw-rw-r--. 1 gpadmin gpadmin  5362 Aug  7 13:21 gpstart_20130807.log
-rw-rw-r--. 1 gpadmin gpadmin  5362 Aug 19 09:10 gpstart_20130819.log
-rw-rw-r--. 1 gpadmin gpadmin  5362 Aug 31 19:27 gpstart_20130831.log
-rw-rw-r--. 1 gpadmin gpadmin  4110 Sep  1 14:44 gpstart_20130901.log
-rw-rw-r--. 1 gpadmin gpadmin  6367 Oct  9 08:44 gpstart_20131009.log
-rw-rw-r--. 1 gpadmin gpadmin 21634 Oct 30 11:14 gpstart_20131030.log
-rw-rw-r--. 1 gpadmin gpadmin  5362 Oct 31 14:23 gpstart_20131031.log
-rw-rw-r--. 1 gpadmin gpadmin 21721 Apr 24  2013 gpstate_20130424.log
-rw-rw-r--. 1 gpadmin gpadmin  2293 Apr 23  2013 gpstop_20130423.log
-rw-rw-r--. 1 gpadmin gpadmin  9410 Apr 24  2013 gpstop_20130424.log
-rw-rw-r--. 1 gpadmin gpadmin  7239 Apr 26  2013 gpstop_20130426.log
-rw-rw-r--. 1 gpadmin gpadmin  8129 Apr 28  2013 gpstop_20130428.log
-rw-rw-r--. 1 gpadmin gpadmin  1258 Apr 29  2013 gpstop_20130429.log
-rw-rw-r--. 1 gpadmin gpadmin   629 May  3  2013 gpstop_20130503.log
-rw-rw-r--. 1 gpadmin gpadmin  1233 May 13  2013 gpstop_20130513.log
-rw-rw-r--. 1 gpadmin gpadmin  1233 May 25 16:24 gpstop_20130525.log
-rw-rw-r--. 1 gpadmin gpadmin  3540 May 31 15:12 gpstop_20130531.log
-rw-rw-r--. 1 gpadmin gpadmin  1233 Jun 10 09:04 gpstop_20130610.log
-rw-rw-r--. 1 gpadmin gpadmin 11275 Jul 24 13:27 gpstop_20130724.log
-rw-rw-r--. 1 gpadmin gpadmin  4823 Jul 25 14:50 gpstop_20130725.log
-rw-rw-r--. 1 gpadmin gpadmin  4823 Aug  1 08:50 gpstop_20130801.log
-rw-rw-r--. 1 gpadmin gpadmin  1233 Aug  2 11:37 gpstop_20130802.log
-rw-rw-r--. 1 gpadmin gpadmin  9645 Aug  3 15:00 gpstop_20130803.log
-rw-rw-r--. 1 gpadmin gpadmin 13008 Aug  7 13:20 gpstop_20130807.log
-rw-rw-r--. 1 gpadmin gpadmin  1233 Aug 19 09:10 gpstop_20130819.log
-rw-rw-r--. 1 gpadmin gpadmin  9505 Aug 31 19:27 gpstop_20130831.log
-rw-rw-r--. 1 gpadmin gpadmin  4822 Sep  1 14:44 gpstop_20130901.log
-rw-rw-r--. 1 gpadmin gpadmin  3540 Oct  9 08:44 gpstop_20131009.log
-rw-rw-r--. 1 gpadmin gpadmin 17812 Oct 30 11:14 gpstop_20131030.log
-rw-rw-r--. 1 gpadmin gpadmin  1233 Oct 31 14:23 gpstop_20131031.log
[gpadmin@sachi gpAdminLogs]$ 

Comments