pgcrypto

Greenplum provides pgcrypto package (PostgreSQL package compiled in Greenplum env) . The pgcrypto package is not installed by default with Greenplum Database, however you can download a pgcrypto package from the EMC Download Center, then use the Greenplum Package Manager (gppkg) to install pgcrypto across your entire cluster.

1. The pgcrypto functions allow database administrators to store certain columns of data in encrypted form. 
2. This adds an extra layer of protection for sensitive data, as data stored in Greenplum Database in encrypted form cannot be read by users who do not
have the encryption key, nor be read directly from the disks.
3. It is important to note that the pgcrypto functions run inside database server. 
4. All the data and passwords move between pgcrypto and the client application in clear-text. 
5. For optimal security, consider also using SSL connections between the client and the Greenplum master server.
6. PgCrypto has various levels of encryption ranging from basic to advanced built-in functions.
7. Encryption makes it difficult to read data but it also comes with a cost of consuming resources to encrypt and decrypt.
8. It is important to pick your encryption strategies based on the sensitivity of the data and performance needs.
9. For Greenplum Database version 4.2 and higher, pgcrypto is available as a package, 
10. You can download from the EMC Download Center and install using the Greenplum Package Manager (gppkg).
==========================================================================================================
==========================================================================================================
==========================================================================================================
To check if pgcrypto libraries are installed on the server:
gppkg -q --all | grep pgcrypto

To Install:
Download the package in a tmp directory.
[gpadmin@mdw tmp]$ ls -ltr | grep pgcrypto
-rw-r----- 1 gpadmin gpadmin 186051 Oct 14 11:03 pgcrypto-ossv1.1_pv1.2_gpdb4.3orca-rhel5-x86_64.gppkg
[gpadmin@mdw tmp]
Source the GPDB environment.[gpadmin@mdw tmp]$ source /usr/local/greenplum-db/greenplum_path.sh
Install the gppkg package using [gpadmin@mdw tmp]$ gppkg -i pgcrypto-ossv1.1_pv1.2_gpdb4.3orca-rhel5-x86_64.gppkg
-----
-----
gppkg:mdw:gpadmin-[INFO]:-pgcrypto-ossv1.1_pv1.2_gpdb4.3orca-rhel5-x86_64.gppkg successfully installed.
Once done you will then need to run the script $GPHOME/share/postgresql/contrib/pgcrypto.sql for the database you want to use pgcrypto on.

To do this run the following command:
psql -d <DBNAME> -f $GPHOME/share/postgresql/contrib/pgcrypto.sql 


To remove the shared libraries:
gppkg -r <Package Name as found in the previous command>

To remove the database function(s):
psql -d <DBNAME> -f $GPHOME/share/postgresql/contrib/uninstall_pgcrypto.sql

==========================================================================================================
==========================================================================================================
==========================================================================================================



Note: The Greenplum Package Manager (gppkg) utility installs pgcrypto and other Greenplum Database extensions, along with any dependencies, on all hosts across a cluster. It will also automatically install extensions on new hosts in the case of system expansion and segment recovery.

Before you install the pgcrypto software package, 

1. Make sure that your Greenplum database is running, 
2. You have sourced greenplum_path.sh, 
3. Following two env variables $MASTER_DATA_DIRECTORY and $GPHOME  are properly set.

1. Download the PostGIS package from the EMC Download Center then copy it to the master host.

Pivotal Advanced Database Services, pgcrypto, Version 1.1.3 Last modified Nov 6, 2013 File Name pgcrypto-1.1.3.0-4609.x86_64.tar.gz
[gpadmin@sachi ~]$ ls -ltr
-rw-r--r--. 1 gpadmin gpadmin   501760 Nov 19 20:07 pgcrypto-1.1.3.0-4609.x86_64.tar
[gpadmin@sachi ~]$ echo $MASTER_DATA_DIRECTORY
/home/gpmaster/gpsne-1
[gpadmin@sachi ~]$ echo $GPHOME 
/usr/local/greenplum-db/.
[gpadmin@sachi ~]$ 
[gpadmin@sachi ~]$ tar -xvf pgcrypto-1.1.3.0-4609.x86_64.tar
./
./share/
./share/postgresql/
./share/postgresql/contrib/
./share/postgresql/contrib/uninstall_pgcrypto.sql
./share/postgresql/contrib/pgcrypto.sql
./lib/
./lib/postgresql/
./lib/postgresql/pgcrypto.so
./pgcrypto_install.sh
[gpadmin@sachi ~]$ 

[gpadmin@sachi ~]$ ls -ltr
total 108164
-r--r--r--. 1 root    root        7008 Oct 17  2012 README_INSTALL
-rwxr-xr-x. 1 root    root    55684023 Oct 17  2012 greenplum-db-4.2.2.4-build-1-CE-RHEL5-x86_64.bin
-rw-r--r--. 1 root    root    54377435 Apr 23  2013 greenplum-db-4.2.2.4-build-1-CE-RHEL5-x86_64.zip
-rwxr-xr-x. 1 gpadmin gpadmin        6 Apr 23  2013 hostlist_singlenode
-rw-rw-r--. 1 gpadmin gpadmin     4515 Jul 24 09:04 gpinitsystem_singlenode
drwxrwxr-x. 3 gpadmin gpadmin     4096 Oct 27 23:56 share
-rwxrwxr-x. 1 gpadmin gpadmin     1983 Oct 27 23:56 pgcrypto_install.sh
drwxrwxr-x. 3 gpadmin gpadmin     4096 Oct 27 23:56 lib
drwxrwxr-x. 2 gpadmin gpadmin     4096 Nov 17 16:29 sachi
-rw-rw-r--. 1 gpadmin gpadmin   144092 Nov 17 18:09 0
-rw-rw-r--. 1 gpadmin gpadmin       81 Nov 18 08:40 table_list.txt
drwxrwxr-x. 2 gpadmin gpadmin     4096 Nov 19 20:00 gpAdminLogs
-rw-r--r--. 1 gpadmin gpadmin   501760 Nov 19 20:07 pgcrypto-1.1.3.0-4609.x86_64.tar

2. Run the install script
[gpadmin@sachi ~]$ ./pgcrypto_install.sh 
<hosts file> not specified.
./pgcrypto_install.sh -f <hosts file>
[gpadmin@sachi ~]$ cat hostlist_singlenode
sachi
=======================================================================================================
[gpadmin@sachi ~]$ cat pgcrypto_install.sh 
#!/bin/sh
#
# Copyright (c) Greenplum Inc 2013. All Rights Reserved.
#
USAGE="$0 -f <hosts file>"

if [ ! -d $GPHOME ]; then
    echo "GPHOME is either not set or is not a directory."
    exit 1
fi

which gpssh > /dev/null
if [ 0 -ne $? ]; then
    echo "gpssh not found in PATH."
    exit 1
fi

which gpscp > /dev/null
if [ 0 -ne $? ]; then
    echo "gpscp not found in PATH."
    exit 1
fi

hosts=""
while getopts f: opt; do
  case $opt in
  f)
      hosts=$OPTARG
      ;;
  esac
done

if [ "$hosts" = "" ]; then
    echo "<hosts file> not specified."
    echo $USAGE
    exit 1
fi

# Validate GPHOME exists on all segments.
cmd="gpssh -f $hosts test -d $GPHOME"
output=$($cmd)
if [ 0 -ne $? ]; then
    echo "Directory $GPHOME not found on one or more segments."
    exit 1
fi
if [[ $output == *ERROR* ]]; then
    echo "Error running gpssh."
    echo "Command: $cmd"
    exit 1
fi

echo "Copying artifacts to master and segments."
# Copy pgcrypto.so to master and segments.
cp lib/postgresql/pgcrypto.so $GPHOME/lib/postgresql
if [ 0 -ne $? ]; then
    echo "Failed to copy artifacts on master."
    exit 1
fi
# SQL scripts are needed only on the master.
cp share/postgresql/contrib/pgcrypto.sql \
    share/postgresql/contrib/uninstall_pgcrypto.sql \
    $GPHOME/share/postgresql/contrib
if [ 0 -ne $? ]; then
    echo "Failed to copy artifacts on master."
    exit 1
fi

cmd="gpscp -f $hosts lib/postgresql/pgcrypto.so =:$GPHOME/lib/postgresql"
output=$($cmd)
if [ 0 -ne $? ]; then
    echo "Failed to copy artifacts to one or more segments."
    # Clean up.
    output=$(gpssh -f $hosts rm -f $GPHOME/lib/postgresql/pgcrypto.so)
    exit 1
fi
if [[ $output == *ERROR* ]]; then
    echo "Error running gpscp."
    echo "Command: $cmd"
    exit 1
fi

echo "Creating pgcrypto functions."
psql -d template1 -f share/postgresql/contrib/pgcrypto.sql
if [ 0 -ne $? ]; then
    echo "Failed to create pgcrypto functions."
    exit 1
fi
=======================================================================================================
Run the 
[gpadmin@sachi ~]$ ./pgcrypto_install.sh -f hostlist_singlenode
gpadmin@sachi's password: 

[gpadmin@sachi ~]$ ./pgcrypto_install.sh
<hosts file> not specified.
./pgcrypto_install.sh -f <hosts file>
[gpadmin@sachi ~]$ ./pgcrypto_install.sh -f hostlist_singlenode
Copying artifacts to master and segments.
Creating pgcrypto functions.
SET
psql:share/postgresql/contrib/pgcrypto.sql:9: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:14: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:19: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:24: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:29: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:34: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:39: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:44: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:49: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:54: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:59: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:64: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:72: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:77: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:85: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:90: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:98: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:103: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:111: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:116: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:124: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:129: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:137: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:142: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:150: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:155: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:163: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:168: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:176: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:181: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:189: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:197: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
psql:share/postgresql/contrib/pgcrypto.sql:202: ERROR:  incompatible library "/usr/local/greenplum-db-4.2.2.4/lib/postgresql/pgcrypto.so": version mismatch (dfmgr.c:360)
DETAIL:  Server version is Greenplum 4.2, library is Product(3209) 4.2.
pgcrypto functions are now ready to use.
[gpadmin@sachi ~]$ 

Make sure that install ends successfully

Enabling PostGIS Support
You must enable pgcrypto support for each database that requires its usage. This is done by running the pgcrypto.sql. pgcrypto.sql contains all the pgcrypto functions.
psql -d dbname -f $GPHOME/share/postgresql/contrib/pgcrypto.sql

Uninstalling pgcrypto
Remove pgcrypto Support
For each database on which you enabled pgcrypto support, you need to execute the following:
psql -d dbname -f $GPHOME/share/postgresql/contrib/uninstall_pgcrypto.sql

Uninstall the Software Package
You can uninstall the pgcrypto software using the Greenplum Package Manager (gppkg), as follows: gppkg -r pgcrypto-1.0
==========================================================================================================
==========================================================================================================
==========================================================================================================
[gpadmin@gpdb-sandbox ~]$ gppkg -h
COMMAND NAME: gppkg
Installs Greenplum Database extensions such as pgcrypto, PL/R, PL/Java, PL/Perl, MADlib and PostGIS, along with their dependencies, across an entire cluster.
*****************************************************
SYNOPSIS
*****************************************************
gppkg [-i <package>| -u <package> | -r <name-version> | -c] 
[-d <master_data_directory>] [-a] [-v]

gppkg --migrate <GPHOME_1> <GPHOME_2> [-a] [-v]
gppkg [-q | --query] query_option
gppkg -? | --help | -h
gppkg --version

*****************************************************
DESCRIPTION
*****************************************************
The Greenplum Package Manager (gppkg) utility installs Greenplum Database extensions, including any dependencies,on all hosts across a cluster. It will also automatically 
install database extensions on new hosts in the case of system expansion and segment recovery.First, download one or more of the available packages from the EMC Download Center (https://emc.subscribenet.com) then copy it to the master host. Use the Greenplum Package Manager to install each package using the options described below.

NOTE: After a major upgrade to Greenplum Database, you must download and install all extensions again.

The following packages are available for download from the
EMC Download Center.
* PostGIS
* PL/Java
* PL/R
* PL/Perl
* MADlib
* Pgcrypto

Note that Greenplum Package Manager installation files for extension packages may release outside of standard Database release cycles. Therefore, for the latest install and configuration information regarding any supported database package/extension, go to the Support site (https://support.emc.com) and download Primus Article 288189
(https://support.emc.com/search/#text=emc288189) from our knowledge base. 
*****************************************************
OPTIONS
*****************************************************

Options
-a (do not prompt)
 Do not prompt the user for confirmation.

-c | --clean
 Reconciles the package state of the cluster to match the  state of the master host. Running this option after a failed  or partial install/uninstall ensures that the package 
 installation state is consistent across the cluster.

 -d <master_data_directory>
 The master data directory. If not specified, the value set  for $MASTER_DATA_DIRECTORY will be used.
 
-i <package> | --install=<package>
 Installs the given package. This includes any pre/post  installation steps and installation of any dependencies.
 
--migrate <GPHOME_1> <GPHOME_2>
 Migrates packages from a separate $GPHOME. Carries over  packages from one version of Greenplum Database to another.
 For example: 
   gppkg --migrate /usr/local/greenplum-db-4.2.0.1 
   /usr/local/greenplum-db-4.2.1.0

 This option is automatically invoked by the installer  during minor upgrades. This option is given here for  cases when the user wants to migrate packages 
 manually. Migration can only proceed if gppkg is executed  from the installation directory to which packages  are being migrated. That is, GPHOME_2 must match 
 the $GPHOME from which the currently executing  gppkg is being run. 

 
-q | --query query_option
 Provides information specified by query_option about the  installed packages. Only one query_option can be specified  at a time. The following table lists the possible values 
 for query_option. <package_file> is the name of a package.

 query_option                  Returns
 ------------                         ------
 <package_file>              Whether the specified package is installed.
 --info <package_file>     The name, version, and other information about the specified package.
 --list <package_file>     The file contents of the specified package.
 --all                              List of all installed packages.

-r <name>-<version> | --remove=<name>-<version>
 Removes the specified package.

 -u <package> | --update=<package>
 Updates the given package.
 
 WARNING: The process of updating a package includes removing all  previous versions of the system objects related to the package. For  example, previous versions of shared libraries are removed.  After the update process, a database function will fail when it is  called if the function references a package file that has been removed. 

--version 
 Displays the version of this utility.

-v | --verbose
 Sets the logging level to verbose.

-? | -h | --help
 Displays the online help.

==========================================================================================================
==========================================================================================================
==========================================================================================================



Comments