Installing Greenplum Chorus
Installing Chorus
You can install Chorus 2.4:
1. on the Standby Master of a Greenplum DCA.
2. on the optional DIA module of a Greenplum DCA.
3. on any Linux server with an Intel Pentium Pro compatible (P3/Athlon and above) CPU and 8GB of RAM.
Greenplum recommends 500GB of free disk space for production level usage. One GB is sufficient for trial usage.
Pre-install checklist
1. Check (with, for example, cat/etc/redhat-release) that your server meets one of the following operating system requirements:
• Red Hat Enterprise Linux 5.5, 5.7, 6.2 (64 bit)
• CentOS 5.5, 5.7, 6.2 (64 bit)
• SuSE Linux Enterprise Server 11 (64 bit)
• OSX Lion x86_64
2. Verify (with java -version) that you have JRE 1.6.0_21 or later installed on your server. JRE 1.7 is not supported.
Note: If you are installing on a DCA, a correct version of Java will have been pre-installed under /usr/java.
3. Check that the variable, JAVA_HOME, is set correctly on your server. For example, if the java executable is in /usr/bin/java then set JAVA_HOME=/usr,
if the java executable is under /usr/java/bin/java, then set JAVA_HOME=/usr/java .
4. Verify that you have one of the following supported browsers running on your client:
• Firefox 17.0 or later
• Google Chrome 23 or later.
• Internet Explorer 8.0 with Google Chrome Frame
• Internet Explorer 9.0 (Google Chrome Frame not required)
Note: IE 9 can be made to simulate IE 7 or IE 8 in its “compatibility mode.”
Chorus does not work with IE 7 or 8 (without Chrome frame), so you must disable compatibility mode. To do this:
a. Press the Alt key to open the IE9 menu bar.
b. Choose the Tools menu.
c. If Compatibility View is unchecked, do nothing.
5. Edit the pg_hba.conf file for any Greenplum Database instances that will be connected to Chorus so that the GPDB instance will accept connections from all users of the Chorus server. See the Greenplum Database Administrator Guide for how to do this.
6. If you want to use Chorus to facilitate creating external tables that point to Hadoop, you merely need to configure Hadoop to work with the Greenplum Database (GPDB). See the Greenplum Database Administrator Guide for instructions on setting up Hadoop with GPDB. Chorus does not require an installation of Hadoop, however.
===========================================================================================
[root@sachi sachi]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.1 (Santiago)
[root@sachi sachi]# which hava
/usr/bin/which: no hava in (/u01/app/oracle/product/11.2.0/db_1/bin:/usr/sbin:/u01/app/oracle/product/11.2.0/db_1/bin:/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/sachi/bin:/home/sachi/bin:/home/sachi/bin:/home/sachi/bin)
[root@sachi sachi]# java -version
java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.11) (rhel-1.61.1.11.11.el6_4-x86_64)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)
[root@sachi sachi]# echo $JAVA_HOME
/usr
[root@sachi sachi]#
===========================================================================================
Pre-install setup
1. Make sure that you have met the requirements listed in “Pre-install checklist” .
2. If you are installing on a DCA, use PuTTY to establish an ssh connection to the GPDB Standby Master or DIA Module.
3. Create the user, chorus, at the shell prompt:
# useradd chorus
# groupadd chorus
# passwd chorus
Note: When you enter the command passwd chorus you are asked for the password: for a DCA installation, choose chorus; for a non-DCA installation, you can choose anything you like.
Choosing chorus as the password will bring up a message (as in the example below)
about not using a dictionary word. chorus will be accepted, however, after you enter it a second time. Here is an example:
[root@smdw /]# passwd chorus
Changing password for user chorus.
New UNIX password:<Enter chorus here>
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:<Enter chorus again>
passwd: all authentication tokens updated successfully.
===========================================================================================
[root@sachi sachi]# useradd chorus
[root@sachi sachi]# passwd chorus
Changing password for user chorus.
New password:
BAD PASSWORD: it is based on a dictionary word
BAD PASSWORD: is too simple
Retype new password:
passwd: all authentication tokens updated successfully.
[root@sachi sachi]#
===========================================================================================
[root@smdw /]#
Although it doesn’t tell you, chorus has been accepted as the password.
4. Switch to user chorus:
# su - chorus
5. Update your .bashrc file:
1. Open your text editor:
$ vi ~/.bashrc
2. Add the following line to your .bashrc file to set the JAVA_HOME variable:
export JAVA_HOME=/usr/java/latest
3. Close the .bashrc file and source it to activate the changes:
$ source ~/.bashrc
4. Return to root:
$ exit
6. Create installation and data directories (as root user):
1. Create the path for the installation binaries:
# mkdir -p /usr/local/greenplum-chorus
# chown -R chorus:chorus /usr/local/greenplum-chorus
2. Create the path for shared data:
# mkdir -p /data/greenplum-chorus
# chown -R chorus:chorus /data/greenplum-chorus
/usr/local/greenplum-chorus and /data/greenplum-chorus are the directories that will be suggested to you when running the installation script. You can substitute any directories of your choice as long as they are owned by the chorus user.
===========================================================================================
[root@sachi local]# mkdir -p /usr/local/greenplum-chorus
[root@sachi local]# chown -R chorus:chorus /usr/local/greenplum-chorus
[root@sachi disk3]# mkdir -p /disk3/greenplum-chorus
[root@sachi disk3]# chown -R chorus:chorus /disk3/greenplum-chorus
===========================================================================================
Note: Greenplum recommends 500GB of free disk space for production level usage. You can run the df -h command as root to see the free space you have on your mounted file systems. 1GB is sufficient for trial usage.
Important: If you are installing on a DCA, skip steps 7 through 9.
7. Set the following parameters in /etc/security/limits.conf:
soft nofile 65536
hard nofile 65536
soft nproc 131072
hard nproc 131072
8. Set the following parameters in /etc/sysctl.conf:
kernel.shmmax = 500000000
kernel.shmall = 4000000000
9. Restart the server if you made changes to the configuration parameters in steps 7 and 8.
Download and Install Install Greenplum Chorus 2.4
1. Go to the EMC Download Center to download the Greenplum Chorus installation package. The package will be in the form greenplum-chorus-2.4.0.0.<build number>-<sha>.zip where sha is a hash that maps to a specific code commit. For example, for the first release of Chorus 2.4 the file is greenplum-chorus-2.4.0.0.1549-9d20ac862.zip
2. Save the package to a folder where the chorus user has write privileges. In the case of a DCA v1 installation, this folder should be on the DCA Standby Master in
the /home/chorus directory. For a DCA v2 UAP Edition, this folder should be on the DIA module in the /home/chorus directory.
3. Run MD5 on the binary. This generates a string which you can compare to the value listed on Support Zone in order to verify that you have downloaded the correct file. For example, running
# md5sum ~/greenplum-chorus-2.4.0.0.1549-9d20ac862.zip
on the DCA smdw could return MD5
greenplum-chorus-2.4.0.0.1549-9d20ac862.zip=935d82688591bf0e6f0ba05dc5837fd3
You can compare 935d82688591bf0e6f0ba05dc5837fd3 with the value listed on Support Zone. If the values match, you have downloaded the correct file.
===========================================================================================
Greenplum Chorus 2.4.0.0
Patch
No
File Name: greenplum-chorus-2.4.0.0.2953-0976b1fe7.zip
Download File Size: 213,099,217 bytes (203.2 MB)
MD5 Signature: 1230b3a268681fbc6c630ffcbe9cd215
[root@sachi chorus]# md5sum greenplum-chorus-2.4.0.0.2953-0976b1fe7.zip
1230b3a268681fbc6c630ffcbe9cd215 greenplum-chorus-2.4.0.0.2953-0976b1fe7.zip
[root@sachi chorus]#
1230b3a268681fbc6c630ffcbe9cd215 ---> Exact match
===========================================================================================
4. You must unzip the installation package greenplum-chorus-2.4.0.0.1549-9d20ac862.zip to obtain greenplum-chorus-2.4.0.0.1549-9d20ac862.sh. This file is a
self-extracting script that contains the following components.
• Chorus code
• PostgreSQL database package
As root, change to the /home/gpadmin directory. Then run # chmod +x ./greenplum-chorus-2.4.0.0.1549-9d20ac862.sh
Running this command gives you binary execution privileges.
5. Log in as the chorus user and execute the installer:
$ ./greenplum-chorus-2.4.0.0.1549-9d20ac862.sh
6. Provide the installer with the following information to continue the process:
a. Type y to accept the license agreement; otherwise the installer will exit.
b. Provide the correct directory if you want the installation binaries at a location different from /usr/local/greenplum-chorus
c. Provide the correct directory if you want shared data at a location different from /data/greenplum-chorus
Ensure that you have adequate disk space for shared data; Greenplum recommends a minimum of 500 GB. You can run the df -h command as root to see the free space you have on your mounted file systems.
7. The installer validates operating system compatibility and displays an error message listing the expected operating systems if your OS is not one of them. You can respond to the error message by choosing one of the listed OS that is equivalent to your OS.
8. You are prompted to enter your passphrase, which can be any combination of alphanumeric characters. This will be used to generate a secret key to be used for
recovering passwords from the GPDB. Write it down and keep it in a safe place!
Note: The secret key is kept (encrypted) in a file named secret.key,located under the shared data directory. When finished, the installer exits.
===========================================================================================
CLCK WRP Rev 20090813
Do you accept the terms above? [y]: y
Please enter the full path to the Chorus installation directory [/usr/local/greenplum-chorus]:
Please enter the full path to the Chorus data directory (recommended 500GB) [/data/greenplum-chorus]: /disk3/greenplum-chorus
Data path = /disk3/greenplum-chorus
Could not detect your Linux version. Please select one of the following:
[1] - RedHat (CentOS/RHEL) 5.5 or compatible
[2] - RedHat (CentOS/RHEL) 6.2 or compatible
[3] - SuSE Enterprise Linux Server 11 or compatible
[4] - Abort install
: 2
Installing Chorus version 2.4.0.0.2953-0976b1fe7 to /usr/local/greenplum-chorus
Copying files into /usr/local/greenplum-chorus...
Configuring secret key...
Enter optional passphrase to generate a recoverable secret key for encrypting passwords. By default, a random key will be generated. []: Gt13355
Configuring secret token...
...done.
Extracting postgres...
...done.
Creating database...
Initializing database...
Running rake db:create db:migrate db:seed enqueue_reindex
...done.
...done.
Installation completed.
To start Chorus, run the following commands:
source /usr/local/greenplum-chorus/chorus_path.sh
chorus_control.sh start
[chorus@sachi ~]$
[chorus@sachi ~]$ cat /usr/local/greenplum-chorus/chorus_path.sh
export CHORUS_HOME=/usr/local/greenplum-chorus
export PATH=$PATH:$CHORUS_HOME
export PGPASSFILE=$CHORUS_HOME/.pgpass
[chorus@sachi ~]$ source /usr/local/greenplum-chorus/chorus_path.sh
[chorus@sachi ~]$
[chorus@sachi ~]$ chorus_control.sh
chorus_control.sh is a utility to start, stop, restart, or monitor the Chorus services.
Usage:
chorus_control.sh start [services] start services
chorus_control.sh stop [services] stop services
chorus_control.sh restart [services] stop and start services
chorus_control.sh monitor [services] monitor and restart services as needed
chorus_control.sh backup [-d dir] [-r days] backup Chorus data
chorus_control.sh restore [file] restore Chorus data
The following services are available: postgres, workers, scheduler, solr, webserver.
If no services are specified on the command line, chorus_control.sh manages all services.
Examples:
chorus_control.sh start start all services
chorus_control.sh stop stop all services
chorus_control.sh restart restart all services
chorus_control.sh monitor monitor all services
chorus_control.sh start postgres solr start specific services
chorus_control.sh stop scheduler workers stop specific services
chorus_control.sh restart webserver restart specific services
chorus_control.sh monitor workers monitor specific services
chorus_control.sh backup -d backup_dir backup Chorus data
chorus_control.sh restore backup_filename restore Chorus data
[chorus@sachi ~]$
[chorus@sachi ~]$ chorus_control.sh start
Setting maximum database connections to 100
[production] postgres started as pid 28274
[production] Worker started as pid 28368
[production] Scheduler started as pid 28465
[production] Solr started as pid 28546 on port 8983
[production] Writing nginx config...
[production] nginx started as pid 28712
[production] updating jetty config...
[production] starting jetty...
[chorus@sachi ~]$
===========================================================================================
9. Prior to starting the Chorus server, you should review the contents of the chorus.properties file. See Chapter 3, “Configuring Greenplum Chorus 2.4”. Also, Greenplum recommends that Chorus is configured with an ssl certificate (see “To generate an SSL certificate with OpenSSL” ).
10.To start the Chorus server, do the following as user chorus:
$ source /usr/local/greenplum-chorus/chorus_path.sh
$ chorus_control.sh start
Verify your Greenplum Chorus installation
Make sure your external network can use Greenplum Chorus with these steps.
1. From an external server, log into Greenplum Chorus:
http://<external IP address smdw>:8080 (DCA installation)
http://<external IP address DIA>:8080 (DIA installation)
http://<external IP address>:8080 (non-DCA installation)
2. Log into Greenplum Chorus with the user name chorusadmin and the password secret
3. Ensure Greenplum Chorus loads in the browser.
Note: Greenplum recommends that you change your username and password after you have verified that your installation works.
Important: After installing Chorus and verifying the installation, you should set up a daily backup (see “Backing up Greenplum Chorus” and a log rotation (see
“Logrotate” .
Default username: chorusadmin
Default password:
secret