pgcrypto

Greenplum provides pgcrypto package (PostgreSQL package compiled in Greenplum env) . The pgcrypto package is not installed by default with Greenplum Database, however you can download a pgcrypto package from the EMC Download Center, then use the Greenplum Package Manager (gppkg) to install pgcrypto across your entire cluster.

1. The pgcrypto functions allow database administrators to store certain columns of data in encrypted form. 

2. This adds an extra layer of protection for sensitive data, as data stored in Greenplum Database in encrypted form cannot be read by users who do not

have the encryption key, nor be read directly from the disks.

3. It is important to note that the pgcrypto functions run inside database server. 

4. All the data and passwords move between pgcrypto and the client application in clear-text. 

5. For optimal security, consider also using SSL connections between the client and the Greenplum master server.

6. PgCrypto has various levels of encryption ranging from basic to advanced built-in functions.

7. Encryption makes it difficult to read data but it also comes with a cost of consuming resources to encrypt and decrypt.

8. It is important to pick your encryption strategies based on the sensitivity of the data and performance needs.

9. For Greenplum Database version 4.2 and higher, pgcrypto is available as a package, 

10. You can download from the EMC Download Center and install using the Greenplum Package Manager (gppkg).

==========================================================================================================

==========================================================================================================

==========================================================================================================

To check if pgcrypto libraries are installed on the server:

gppkg -q --all | grep pgcrypto

To Install:

Download the package in a tmp directory.

[gpadmin@mdw tmp]$ ls -ltr | grep pgcrypto

-rw-r----- 1 gpadmin gpadmin 186051 Oct 14 11:03 pgcrypto-ossv1.1_pv1.2_gpdb4.3orca-rhel5-x86_64.gppkg

[gpadmin@mdw tmp]

Source the GPDB environment.[gpadmin@mdw tmp]$ source /usr/local/greenplum-db/greenplum_path.sh

Install the gppkg package using [gpadmin@mdw tmp]$ gppkg -i pgcrypto-ossv1.1_pv1.2_gpdb4.3orca-rhel5-x86_64.gppkg

-----

-----

gppkg:mdw:gpadmin-[INFO]:-pgcrypto-ossv1.1_pv1.2_gpdb4.3orca-rhel5-x86_64.gppkg successfully installed.

Once done you will then need to run the script $GPHOME/share/postgresql/contrib/pgcrypto.sql for the database you want to use pgcrypto on.

To do this run the following command:

psql -d <DBNAME> -f $GPHOME/share/postgresql/contrib/pgcrypto.sql 

To remove the shared libraries:

gppkg -r <Package Name as found in the previous command>

To remove the database function(s):

psql -d <DBNAME> -f $GPHOME/share/postgresql/contrib/uninstall_pgcrypto.sql

==========================================================================================================

==========================================================================================================

==========================================================================================================

Note: The Greenplum Package Manager (gppkg) utility installs pgcrypto and other Greenplum Database extensions, along with any dependencies, on all hosts across a cluster. It will also automatically install extensions on new hosts in the case of system expansion and segment recovery.

Before you install the pgcrypto software package, 

1. Make sure that your Greenplum database is running, 

2. You have sourced greenplum_path.sh, 

3. Following two env variables $MASTER_DATA_DIRECTORY and $GPHOME  are properly set.

1. Download the PostGIS package from the EMC Download Center then copy it to the master host.

Pivotal Advanced Database Services, pgcrypto, Version 1.1.3 Last modified Nov 6, 2013 File Name pgcrypto-1.1.3.0-4609.x86_64.tar.gz

[gpadmin@sachi ~]$ ls -ltr

-rw-r--r--. 1 gpadmin gpadmin   501760 Nov 19 20:07 pgcrypto-1.1.3.0-4609.x86_64.tar

[gpadmin@sachi ~]$ echo $MASTER_DATA_DIRECTORY

/home/gpmaster/gpsne-1

[gpadmin@sachi ~]$ echo $GPHOME 

/usr/local/greenplum-db/.

[gpadmin@sachi ~]$ 

[gpadmin@sachi ~]$ tar -xvf pgcrypto-1.1.3.0-4609.x86_64.tar

./

./share/

./share/postgresql/

./share/postgresql/contrib/

./share/postgresql/contrib/uninstall_pgcrypto.sql

./share/postgresql/contrib/pgcrypto.sql

./lib/

./lib/postgresql/

./lib/postgresql/pgcrypto.so

./pgcrypto_install.sh

[gpadmin@sachi ~]$ 

[gpadmin@sachi ~]$ ls -ltr

total 108164

-r--r--r--. 1 root    root        7008 Oct 17  2012 README_INSTALL

-rwxr-xr-x. 1 root    root    55684023 Oct 17  2012 greenplum-db-4.2.2.4-build-1-CE-RHEL5-x86_64.bin

-rw-r--r--. 1 root    root    54377435 Apr 23  2013 greenplum-db-4.2.2.4-build-1-CE-RHEL5-x86_64.zip

-rwxr-xr-x. 1 gpadmin gpadmin        6 Apr 23  2013 hostlist_singlenode

-rw-rw-r--. 1 gpadmin gpadmin     4515 Jul 24 09:04 gpinitsystem_singlenode

drwxrwxr-x. 3 gpadmin gpadmin     4096 Oct 27 23:56 share

-rwxrwxr-x. 1 gpadmin gpadmin     1983 Oct 27 23:56 pgcrypto_install.sh

drwxrwxr-x. 3 gpadmin gpadmin     4096 Oct 27 23:56 lib

drwxrwxr-x. 2 gpadmin gpadmin     4096 Nov 17 16:29 sachi

-rw-rw-r--. 1 gpadmin gpadmin   144092 Nov 17 18:09 0

-rw-rw-r--. 1 gpadmin gpadmin       81 Nov 18 08:40 table_list.txt

drwxrwxr-x. 2 gpadmin gpadmin     4096 Nov 19 20:00 gpAdminLogs

-rw-r--r--. 1 gpadmin gpadmin   501760 Nov 19 20:07 pgcrypto-1.1.3.0-4609.x86_64.tar

2. Run the install script

[gpadmin@sachi ~]$ ./pgcrypto_install.sh 

<hosts file> not specified.

./pgcrypto_install.sh -f <hosts file>

[gpadmin@sachi ~]$ cat hostlist_singlenode

sachi

=======================================================================================================

[gpadmin@sachi ~]$ cat pgcrypto_install.sh 

#!/bin/sh

#

# Copyright (c) Greenplum Inc 2013. All Rights Reserved.

#

USAGE="$0 -f <hosts file>"

if [ ! -d $GPHOME ]; then

    echo "GPHOME is either not set or is not a directory."

    exit 1

fi

which gpssh > /dev/null

if [ 0 -ne $? ]; then

    echo "gpssh not found in PATH."

    exit 1

fi

which gpscp > /dev/null

if [ 0 -ne $? ]; then

    echo "gpscp not found in PATH."

    exit 1

fi

hosts=""

while getopts f: opt; do

  case $opt in

  f)

      hosts=$OPTARG

      ;;

  esac

done

if [ "$hosts" = "" ]; then

    echo "<hosts file> not specified."

    echo $USAGE

    exit 1

fi

# Validate GPHOME exists on all segments.

cmd="gpssh -f $hosts test -d $GPHOME"

output=$($cmd)

if [ 0 -ne $? ]; then

    echo "Directory $GPHOME not found on one or more segments."

    exit 1

fi

if [[ $output == *ERROR* ]]; then

    echo "Error running gpssh."

    echo "Command: $cmd"

    exit 1

fi

echo "Copying artifacts to master and segments."

# Copy pgcrypto.so to master and segments.

cp lib/postgresql/pgcrypto.so $GPHOME/lib/postgresql

if [ 0 -ne $? ]; then

    echo "Failed to copy artifacts on master."

    exit 1

fi

# SQL scripts are needed only on the master.

cp share/postgresql/contrib/pgcrypto.sql \

    share/postgresql/contrib/uninstall_pgcrypto.sql \

    $GPHOME/share/postgresql/contrib

if [ 0 -ne $? ]; then

    echo "Failed to copy artifacts on master."

    exit 1

fi

cmd="gpscp -f $hosts lib/postgresql/pgcrypto.so =:$GPHOME/lib/postgresql"

output=$($cmd)

if [ 0 -ne $? ]; then

    echo "Failed to copy artifacts to one or more segments."

    # Clean up.

    output=$(gpssh -f $hosts rm -f $GPHOME/lib/postgresql/pgcrypto.so)

    exit 1

fi

if [[ $output == *ERROR* ]]; then

    echo "Error running gpscp."

    echo "Command: $cmd"

    exit 1

fi

echo "Creating pgcrypto functions."

psql -d template1 -f share/postgresql/contrib/pgcrypto.sql

if [ 0 -ne $? ]; then

    echo "Failed to create pgcrypto functions."

    exit 1

fi

=======================================================================================================

Run the 

[gpadmin@sachi ~]$ ./pgcrypto_install.sh -f hostlist_singlenode

gpadmin@sachi's password: 

[gpadmin@sachi ~]$ ./pgcrypto_install.sh

<hosts file> not specified.

./pgcrypto_install.sh -f <hosts file>

[gpadmin@sachi ~]$ ./pgcrypto_install.sh -f hostlist_singlenode

Copying artifacts to master and segments.

Creating pgcrypto functions.

SET

psql:share/postgresql/contrib/pgcrypto.sql:9: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:14: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:19: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:24: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:29: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:34: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:39: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:44: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:49: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:54: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:59: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:64: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:72: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:77: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:85: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:90: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:98: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:103: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:111: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:116: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:124: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:129: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:137: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:142: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:150: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:155: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:163: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:168: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:176: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:181: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:189: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:197: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

psql:share/postgresql/contrib/pgcrypto.sql:202: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)

DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.

pgcrypto functions are now ready to use.

[gpadmin@sachi ~]$ 

Make sure that install ends successfully

Enabling PostGIS Support

You must enable pgcrypto support for each database that requires its usage. This is done by running the pgcrypto.sql. pgcrypto.sql contains all the pgcrypto functions.

psql -d dbname -f $GPHOME/share/postgresql/contrib/pgcrypto.sql

Uninstalling pgcrypto

Remove pgcrypto Support

For each database on which you enabled pgcrypto support, you need to execute the following:

psql -d dbname -f $GPHOME/share/postgresql/contrib/uninstall_pgcrypto.sql

Uninstall the Software Package

You can uninstall the pgcrypto software using the Greenplum Package Manager (gppkg), as follows: gppkg -r pgcrypto-1.0

==========================================================================================================

==========================================================================================================

==========================================================================================================

[gpadmin@gpdb-sandbox ~]$ gppkg -h

COMMAND NAME: gppkg

Installs Greenplum Database extensions such as pgcrypto, PL/R, PL/Java, PL/Perl, MADlib and PostGIS, along with their dependencies, across an entire cluster.

*****************************************************

SYNOPSIS

*****************************************************

gppkg [-i <package>| -u <package> | -r <name-version> | -c] 

[-d <master_data_directory>] [-a] [-v]

gppkg --migrate <GPHOME_1> <GPHOME_2> [-a] [-v]

gppkg [-q | --query] query_option

gppkg -? | --help | -h

gppkg --version

*****************************************************

DESCRIPTION

*****************************************************

The Greenplum Package Manager (gppkg) utility installs Greenplum Database extensions, including any dependencies,on all hosts across a cluster. It will also automatically 

install database extensions on new hosts in the case of system expansion and segment recovery.First, download one or more of the available packages from the EMC Download Center (https://emc.subscribenet.com) then copy it to the master host. Use the Greenplum Package Manager to install each package using the options described below.

NOTE: After a major upgrade to Greenplum Database, you must download and install all extensions again.

The following packages are available for download from the

EMC Download Center.

* PostGIS

* PL/Java

* PL/R

* PL/Perl

* MADlib

* Pgcrypto

Note that Greenplum Package Manager installation files for extension packages may release outside of standard Database release cycles. Therefore, for the latest install and configuration information regarding any supported database package/extension, go to the Support site (https://support.emc.com) and download Primus Article 288189

(https://support.emc.com/search/#text=emc288189) from our knowledge base. 

*****************************************************

OPTIONS

*****************************************************

Options

-a (do not prompt)

 Do not prompt the user for confirmation.

-c | --clean

 Reconciles the package state of the cluster to match the  state of the master host. Running this option after a failed  or partial install/uninstall ensures that the package 

 installation state is consistent across the cluster.

 -d <master_data_directory>

 The master data directory. If not specified, the value set  for $MASTER_DATA_DIRECTORY will be used.

 

-i <package> | --install=<package>

 Installs the given package. This includes any pre/post  installation steps and installation of any dependencies.

 

--migrate <GPHOME_1> <GPHOME_2>

 Migrates packages from a separate $GPHOME. Carries over  packages from one version of Greenplum Database to another.

 For example: 

   gppkg --migrate /usr/local/greenplum-db-4.2.0.1 

   /usr/local/greenplum-db-4.2.1.0

 This option is automatically invoked by the installer  during minor upgrades. This option is given here for  cases when the user wants to migrate packages 

 manually. Migration can only proceed if gppkg is executed  from the installation directory to which packages  are being migrated. That is, GPHOME_2 must match 

 the $GPHOME from which the currently executing  gppkg is being run. 

 

-q | --query query_option

 Provides information specified by query_option about the  installed packages. Only one query_option can be specified  at a time. The following table lists the possible values 

 for query_option. <package_file> is the name of a package.

 query_option                  Returns

 ------------                         ------

 <package_file>              Whether the specified package is installed.

 --info <package_file>     The name, version, and other information about the specified package.

 --list <package_file>     The file contents of the specified package.

 --all                              List of all installed packages.

-r <name>-<version> | --remove=<name>-<version>

 Removes the specified package.

 -u <package> | --update=<package>

 Updates the given package.

 

 WARNING: The process of updating a package includes removing all  previous versions of the system objects related to the package. For  example, previous versions of shared libraries are removed.  After the update process, a database function will fail when it is  called if the function references a package file that has been removed. 

--version 

 Displays the version of this utility.

-v | --verbose

 Sets the logging level to verbose.

-? | -h | --help

 Displays the online help.

==========================================================================================================

==========================================================================================================

==========================================================================================================