Finding the status of running greenplum database system

Post date: Nov 18, 2013 1:13:41 AM

gpstate utility provides the status of each individual component of a Greenplum Database system, including primary segments,mirror segments, master, and standby master. using $gpstate -e command you can find the state of the greenplum database system.

[gpadmin@sachi gpsne-1]$ gpstate -e

20131117:19:59:06:012754 gpstate:sachi:gpadmin-[INFO]:-Starting gpstate with args: -e

20131117:19:59:06:012754 gpstate:sachi:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.2.2.4 build 1 Community Edition'

20131117:19:59:07:012754 gpstate:sachi:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28'

20131117:19:59:07:012754 gpstate:sachi:gpadmin-[INFO]:-Obtaining Segment details from master...

20131117:19:59:07:012754 gpstate:sachi:gpadmin-[INFO]:-Physical mirroring is not configured

[gpadmin@sachi gpsne-1]$ gpstate --help

COMMAND NAME: gpstate

Shows the status of a running Greenplum Database system.

*****************************************************

SYNOPSIS

*****************************************************

gpstate [-d <master_data_directory>] [-B <parallel_processes>] 

        [-s | -b | -Q | -e] [-m | -c] [-p] [-i] [-f] 

        [-v | -q] [-l <log_directory>]

gpstate -? | -h | --help

*****************************************************

DESCRIPTION

*****************************************************

The gpstate utility displays information about a running Greenplum Database instance. There is additional information you may want to know about a Greenplum Database system, since it is comprised of multiple PostgreSQL database instances (segments) spanning multiple machines. The gpstate utility provides 

additional status information for a Greenplum Database system, such as:

* Which segments are down.

* Master and segment configuration information (hosts, data directories, etc.).

* The ports used by the system.

* A mapping of primary segments to their corresponding mirror segments.

*****************************************************

OPTIONS

*****************************************************

-b (brief status)

Optional. Display a brief summary of the state of the Greenplum Database system. This is the default option.

-B <parallel_processes> 

The number of segments to check in parallel. If not specified,  the utility will start up to 60 parallel processes depending on how many segment instances it needs to check.

-c (show primary to mirror mappings)

Optional. Display mapping of primary segments to their corresponding mirror segments.

-d <master_data_directory>

Optional. The master data directory. If not specified, the value set for $MASTER_DATA_DIRECTORY will be used.

-e (show segments with mirror status issues)

Show details on primary/mirror segment pairs that have potential issues such as 1) the active segment is running in change tracking mode, meaning a segment is down 2) the active segment is in resynchronization mode, meaning it is catching up changes to the mirror 3) a segment is not in its preferred role, for example a segment that was a primary at system initialization time is now acting as a mirror, meaning you may have one or more segment hosts with unbalanced processing load.

-f (show standby master details)

Display details of the standby master host if configured.

-i (show Greenplum Database version)

Display the Greenplum Database software version information for each instance.

-l <logfile_directory>

The directory to write the log file. Defaults to ~/gpAdminLogs.

-m (list mirrors)

Optional. List the mirror segment instances in the system, their current role, and synchronization status.

-p (show ports)

List the port numbers used throughout the Greenplum Database system.

-q (no screen output)

Optional. Run in quiet mode. Except for warning messages, command output is not displayed on the screen. However, this information is still written to the log file.

-Q (quick status)

Optional. Checks segment status in the system catalog on the master host. Does not poll the segments for status.

-s (detailed status)

Optional. Displays detailed status information for the Greenplum Database system.

-v (verbose output)

Optional. Displays error messages and outputs detailed status and progress information.

-? | -h | --help (help)

 Displays the online help.

*****************************************************

OUTPUT DEFINITIONS FOR DETAIL VIEW

*****************************************************

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

MASTER OUTPUT DATA

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

* Master host - host name of the master

* Master postgres process ID - PID of the master postgres database listener process

* Master data directory - file system location of the master data directory

* Master port - port of the master database listener process

* Master current role - dispatch = regular operating mode utility = maintenance mode 

* Greenplum array configuration type - Standard = one NIC per host Multi-Home = multiple NICs per host

* Greenplum initsystem version - version of Greenplum Database when system was first initialized

* Greenplum current version - current version of Greenplum Database

* Postgres version - version of PostgreSQL that Greenplum Database is based on

* Greenplum mirroring status - physical mirroring, SAN or none

* Master standby - host name of the standby master

* Standby master state - status of the standby master: active or passive

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SEGMENT OUTPUT DATA

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

* Hostname - system-configured host name

* Address - network address host name (NIC name)

* Datadir - file system location of segment data directory

* Port - port number of segment postgres database listener process

* Current Role - current role of a segment: Mirror or Primary 

* Preferred Role - role at system initialization time: Mirror or Primary

* Mirror Status - status of a primary/mirror segment pair:

Synchronized = data is up to date on both 

Resynchronizing = data is currently being copied from one to the other

Change Tracking = segment down and active segment is logging changes

* Change tracking data size - when in Change Tracking mode, the size of the change log file (may grow and shrink as compression is applied)

* Estimated total data to synchronize - when in Resynchronization mode, the estimated size of data left to syncronize

* Data synchronized - when in Resynchronization mode, the estimated size of data that has already been syncronized

* Estimated resync progress with mirror - when in Resynchronization mode, the estimated percentage of completion

* Estimated resync end time - when in Resynchronization mode, the estimated time to complete

* File postmaster.pid - status of postmaster.pid lock file: Found or Missing

* PID from postmaster.pid file - PID found in the postmaster.pid file

* Lock files in /tmp - a segment port lock file for its postgres process is created in /tmp (file is removed when a segment shuts down)

* Active PID - active process ID of a segment

* Master reports status as - segment status as reported in the system catalog:  Up or Down

Database status - status of Greenplum Database to incoming requests: Up, Down, or Suspended. A Suspended state means database activity is temporarily paused while a segment transitions from one state to another.

*****************************************************

EXAMPLES

*****************************************************

Show detailed status information of a Greenplum Database system:

gpstate -s

$ gpstate -s

20131117:20:10:29:014187 gpstate:sachi:gpadmin-[INFO]:-Starting gpstate with args: -s

20131117:20:10:29:014187 gpstate:sachi:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.2.2.4 build 1 Community Edition'

20131117:20:10:29:014187 gpstate:sachi:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28'

20131117:20:10:29:014187 gpstate:sachi:gpadmin-[INFO]:-Obtaining Segment details from master...

20131117:20:10:29:014187 gpstate:sachi:gpadmin-[INFO]:-Gathering data from segments...

.. 

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:--Master Configuration & Status

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Master host                    = sachi

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Master postgres process ID     = 5307

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Master data directory          = /home/gpmaster/gpsne-1

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Master port                    = 5432

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Master current role            = dispatch

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Greenplum initsystem version   = 4.2.2.4 build 1 Community Edition

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Greenplum current version      = PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Postgres version               = 8.2.15

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Master standby                 = No master standby configured

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-Segment Instance Status Report

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Segment Info

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Hostname                          = sachi

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Address                           = sachi

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Datadir                           = /disk1/gpdata1/gpsne0

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Port                              = 40000

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Status

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      PID                               = 5275

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Configuration reports status as   = Up

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Database status                   = Up

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Segment Info

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Hostname                          = sachi

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Address                           = sachi

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Datadir                           = /disk2/gpdata2/gpsne1

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Port                              = 40001

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Status

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      PID                               = 5276

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Configuration reports status as   = Up

20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Database status                   = Up

[gpadmin@sachi gpsne-1]$ 

Do a quick check for down segments in the master host system catalog:

gpstate -Q

[gpadmin@sachi gpsne-1]$ gpstate -Q

20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-Starting gpstate with args: -Q

20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.2.2.4 build 1 Community Edition'

20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28'

20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-Obtaining Segment details from master...

20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:--Quick Greenplum database status from Master instance only

20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------------

20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-# of up segments, from configuration table     = 2

20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------------

Show information about mirror segment instances:

gpstate -m

[gpadmin@sachi gpsne-1]$ gpstate -m

20131117:20:12:02:014537 gpstate:sachi:gpadmin-[INFO]:-Starting gpstate with args: -m

20131117:20:12:02:014537 gpstate:sachi:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.2.2.4 build 1 Community Edition'

20131117:20:12:02:014537 gpstate:sachi:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28'

20131117:20:12:02:014537 gpstate:sachi:gpadmin-[INFO]:-Obtaining Segment details from master...

20131117:20:12:02:014537 gpstate:sachi:gpadmin-[WARNING]:--------------------------------------------------------------

20131117:20:12:02:014537 gpstate:sachi:gpadmin-[WARNING]:-physical mirroring not used

20131117:20:12:02:014537 gpstate:sachi:gpadmin-[WARNING]:--------------------------------------------------------------

Show information about the standby master configuration:

gpstate -f

gpadmin@sachi gpsne-1]$ gpstate -f

20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:-Starting gpstate with args: -f

20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.2.2.4 build 1 Community Edition'

20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28'

20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:-Obtaining Segment details from master...

20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:-Standby master instance not configured

20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--------------------------------------------------------------

20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--gp_master_mirroring table

20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--------------------------------------------------------------

20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--Summary state: Not Configured

20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--Detail state: 

20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--Log time: 

20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--------------------------------------------------------------

Display the Greenplum software version information:

gpstate -i

[gpadmin@sachi gpsne-1]$ gpstate -i

20131117:20:12:45:014658 gpstate:sachi:gpadmin-[INFO]:-Starting gpstate with args: -i

20131117:20:12:45:014658 gpstate:sachi:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.2.2.4 build 1 Community Edition'

20131117:20:12:46:014658 gpstate:sachi:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28'

20131117:20:12:46:014658 gpstate:sachi:gpadmin-[INFO]:-Obtaining Segment details from master...

20131117:20:12:46:014658 gpstate:sachi:gpadmin-[INFO]:-Loading version information

20131117:20:12:47:014658 gpstate:sachi:gpadmin-[INFO]:-   Host    Datadir                  Port    Version                                                                                                                                                                  

20131117:20:12:47:014658 gpstate:sachi:gpadmin-[INFO]:-   sachi   /home/gpmaster/gpsne-1   5432    PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28

20131117:20:12:47:014658 gpstate:sachi:gpadmin-[INFO]:-   sachi   /disk1/gpdata1/gpsne0    40000   PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28

20131117:20:12:47:014658 gpstate:sachi:gpadmin-[INFO]:-   sachi   /disk2/gpdata2/gpsne1    40001   PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28

20131117:20:12:47:014658 gpstate:sachi:gpadmin-[INFO]:-All segments are running the same software version