Eric Radman : a Journal

Building a Custom PostgreSQL Release

While PostgreSQL is a large project, it is remarkably straitfoward to build and test. Here is some of the workflow I have used.

Build from source

./configure --enable-depend --enable-cassert --enable-debug --enable-thread-safety --enable-tap-tests
gmake
gmake world
doas gmake install

To run regressions

gmake -C src/bin check

Functional Testing

Manual testing should not mean manual setup. I found it very valuable to create a simple script for initializing two database servers

 # init_databases.sh
export PATH=/usr/local/pgsql/bin:$PATH

rm -rf /var/postgresql/server{1,2}

initdb -D /var/postgresql/server1
cat >> /var/postgresql/server1/postgresql.conf << EOF
port = 5433
log_min_messages = debug3

hot_standby = on
max_replication_slots = 2
max_wal_senders = 5
hot_standby_feedback = on
wal_log_hints = on
wal_level = hot_standby
EOF

pg_ctl -D /var/postgresql/server1 start -w
PGPORT=5433 createdb
PGPORT=5433 createdb localharvest

PGPORT=5433 pg_basebackup -D /var/postgresql/server2 -P -h localhost
cat >> /var/postgresql/server2/postgresql.conf << EOF
port = 5434
log_min_messages = debug3

hot_standby = on
max_replication_slots = 2
max_wal_senders = 5
hot_standby_feedback = on
wal_log_hints = on
wal_level = hot_standby
EOF

cat > /var/postgresql/server2/recovery.conf << EOF
standby_mode = 'on'
primary_conninfo = 'user=_postgresql host=localhost port=5433'
recovery_target_timeline = 'latest'
recovery_min_apply_delay = '600 s'
EOF

pg_ctl -D /var/postgresql/server1 stop -m smart -w

This is good, but we can take this even further by start development environment for debugging postgresql replication in tmux

#!/bin/sh

session=pg11
editor=${EDITOR:-vim}

export PATH=/usr/local/pgsql/bin:$PATH

server1=/var/postgresql/server1
server2=/var/postgresql/server2

doas -u _postgresql sh -x /tmp/init_databases.sh

# start new session, disconnected
tmux new-session -s "$session" -d

for n in 0 1; do
window=$session:$n
case $n in
    0)
        tmux rename-window -t $window "psql"
        tmux send-keys -t $window "export PATH=/usr/local/pgsql/bin:$PATH" C-m
        tmux send-keys -t $window "doas -u _postgresql pg_ctl -D $server1 -l $server1.log start" C-m
        tmux send-keys -t $window "doas -u _postgresql psql postgresql://localhost:5433" C-m
        tmux send-keys -t $window "SELECT pid,wait_event_type, wait_event,backend_type FROM pg_stat_activity WHERE pid != pg_backend_pid();" C-m
        tmux split-window -t $window
        tmux send-keys -t $window "export PATH=/usr/local/pgsql/bin:$PATH" C-m
        tmux send-keys -t $window "doas -u _postgresql pg_ctl -D $server2 -l $server2.log start" C-m
        tmux send-keys -t $window "doas -u _postgresql psql postgresql://localhost:5434" C-m
        tmux send-keys -t $window "SELECT pid,wait_event_type,wait_event,backend_type FROM pg_stat_activity WHERE pid != pg_backend_pid();" C-m
        tmux select-pane -t $window.0
        tmux split-window -t $window -h
        tmux send-keys -t $window "export PATH=/usr/local/pgsql/bin:$PATH" C-m
        tmux select-pane -t $window.0
        ;;
    1)
        tmux new-window -t $window -n "server.log"
        tmux send-keys -t $window "doas -u _postgresql tail -f /var/postgresql/server1.log" C-m
        tmux split-window -t $window
        tmux send-keys -t $window "doas -u _postgresql tail -f /var/postgresql/server2.log" C-m
        tmux select-pane -t $window.0
        ;;
esac
done

# Switch to first window and attach to session
tmux select-window -t $session:0
tmux -2 attach-session -t "$session"

# destroy session on disconnect in order to maintain a predictable development
# environment
doas -u _postgresql pg_ctl -D $server1 stop -m fast
doas -u _postgresql pg_ctl -D $server2 stop -m fast
tmux kill-session -t "$session"

Building an RPM

For CentOS 7 the list of packages you need is long, mostly to do with building the documentation. Here is the list I install

bison
docbook-dtds
docbook-style-dsssl
docbook-style-xsl
docbook-utils
e2fsprogs-devel
flex
fop
gcc
git
help2man
krb5-devel
libselinux-devel
libuuid-devel
libxslt
libxslt-devel
make
multilib-rpm-config
openjade
openldap-devel
opensp
openssl-devel
pam-devel
perl-ExtUtils-Embed
perl-ExtUtils-MakeMaker
python-devel
readline-devel
rpm-build
systemd-devel
systemtap-sdt-devel
tcl-devel
uuid-devel

For 11 two additional libraries are required

llvm-toolset-7-clang
llvm5.0-devel

I'd recommend using the vanilla spec file from postgresql.org

mkdir -p ~/rpmbuild/{SOURCES,SPECS}
cd ~/rpmbuild/SOURCES
wget "https://download.postgresql.org/pub/repos/yum/srpms/11/redhat/rhel-7Server-x86_64/postgresql11-11.1-1PGDG.rhel7.src.rpm"
rpm2cpio postgresql11-11.1-1PGDG.rhel7.src.rpm | cpio -idmv
rm *.bz2 *.rpm
mv postgresql-11.spec ../SPECS/

Modify ~/rpmbuild/SPECS/postgresql-11.spec to ensure that docs are built by adding the following line

%{__make} -C doc/src/sgml DESTDIR=%{buildroot} all

After developing a patch, checkout the stable branch you would like to update

git checkout REL_11.1
git checkout -b REL_11.1
patch -p1 < ~/patches/delay-reconnect-param.patch
git commit -a -m 'Add recovery_min_apply_delay_reconnect recovery option'

Next generate a source tar from the last commit

package=postgresql-11.1
commit=$(git log --format="%h" -n 1)
git archive --prefix=$package/ $commit | bzip2 > ~/rpmbuild/SOURCES/$package.tar.bz2
sha256sum ~/rpmbuild/SOURCES/$package.tar.bz2 > ~/rpmbuild/SOURCES/$package.tar.bz2.sha256

Then run the build using

cd ~/rpmbuild
rpmbuild --define 'pgmajorversion 11' -bb SPECS/postgresql-11.spec

Last updated on December 31, 2018