Friday, February 05, 2010

Building latest PostgreSQL CVS Head on OpenSolaris

With the talk about PostgreSQL 9.0 alpha 5, I thought it is time for me to try out another CVS head build on OpenSolaris. Of course this time this was on my home desktop which runs OpenSolaris 2009.06
I wanted to download the CVS head. The instructions are there on the wiki page. However before following it I needed to install the CVS on my OpenSolaris instance.

#pkg install SUNWcvs

The using the instructions I created a copy of the cvs repository and created my own project workspace.

I already had the Sun Compilers on my setup. (If not or have an old copy then you can always install or upgrade it as
# pkg install sunstudio
)
So I started my task of creating the new binaries on OpenSolaris. I found it to be bit bumpy.
Here is my configure options with my standard options.
$ ./configure --prefix=$HOME/project CFLAGS="-xO3 -xarch=native -xspace -W0,-Lt -W2,-Rcond_elim -Xa  -xildoff -xc99=none -xCC" --without-readline

However then I hit my first problem to do make.
Need to do using GNU make which was not installed on my desktop.
Back to pkg manager

# pkg install SUNWgmake

Continuing again with make which proceeded and then eventually stopped again due to missing bison. (I wonder why the "configure" script did  not catch that?)

# pkg install SUNWbison

Anyway I started to run make again now that I have installed bison. Strangely it failed again.
Figured it still did not find bison. I had to use ./configure statement again and tried gmake after that which allowed gmake to pick up bison.

However it failed again.

gmake[3]: Entering directory `/export/home/postgres/project/pgsql.project/src/backend/parser'
/usr/bin/bison -d  -o gram.c gram.y
gmake[3]: *** [gram.c] Broken Pipe
gmake[3]: Leaving directory `/export/home/postgres/project/pgsql.project/src/backend/parser'
gmake[2]: *** [parser/gram.h] Error 2

This one was not easy to solve. I thought that probably the bison was buggy and was about to give up. Then I thought I will give it a shot using truss to figure out what is happening

$ truss -f /usr/bin/bison -d -o gram.c gram.y 2> /tmp/bisontruss.txt

Going through that file bisontruss.txt I found:

10451:  fcntl(6, F_DUP2FD, 0x00000001)                  = 1
10451:  close(6)                                        = 0
10451:  execve("/usr/sfw/bin/gm4", 0x08047D00, 0x08047DA0) Err#2 ENOENT

I had no clue what gm4 does, but it is missing.
I used the search feature of OpenSolaris to see if there is a package associated with it.

# pkg search gm4
INDEX      ACTION    VALUE                     PACKAGE
.....
basename   link      usr/sfw/bin/gm4           pkg:/SUNWgm4@1.4.2-0.111

As they say on TV : Yep, there's a pkg for that

# pkg install SUNWgm4

And now back to gmake

Darn another package missing:
gmake[3]: Entering directory `/export/home/postgres/project/pgsql.project/src/backend/bootstrap'
***
ERROR: `flex' is missing on your system. It is needed to create the
file `bootscanner.c'. You can either get flex from a GNU mirror site
or download an official distribution of PostgreSQL, which contains
pre-packaged flex output.
***
# pkg search -r flex
INDEX      ACTION    VALUE                     PACKAGE
....
basename   link      usr/sfw/bin/flex          pkg:/SUNWflexlex@2.5.33-0.111

# pkg install SUNWflexlex

(Of course gmake wont use it immediately till you use the configure statement again)

Finally after a long time (it sure seemed to take a long time) the gmake seemed to hang on preproc.c compilation:

"preproc.y", line 13548: warning: line number in #line directive must be less than or equal to 32767

This is just not my day.
I did a cleanup of the make. At this point I found that I forgot to enable dtrace probes in my configure statement and also since I am using 64-bit kernel, decided to build 64-bit binaries. Retrying with the slightly modified configure still did not solve the loop problem in preproc.y which just runs 100% on the CPU. I left it running for a long while just to see if it finished (till my patience runs out) It did not finish.

I finally updated my sunstudio binaries and retired  gmake. This time it succeeded in finishing the gmake. (Hence I changed my wordings above to reflect install/upgrade sunstudio).

After that a quick gmake install

$ gmake install

and the binaries are ready.
A quick check reflects
$ initdb
$ pg_ctl start -l server.log
$ psql postgres
psql (8.5devel)
Type "help" for help.

postgres=#

For a second I was expecting to see psql(9.0develop) but this is still acceptable. :-)

In Summary, you want to do the following before building the latest PostgreSQL source on OpenSolaris

# pkg install SUNWcvs sunstudio SUNWgmake SUNWbison SUNWgm4 SUNWflexlex

and the configure script
$ ./configure --prefix=$HOME/project CFLAGS="-m64 -xO3  -xarch=native -xspace -W0,-Lt -W2,-Rcond_elim -Xa  -xildoff -xc99=none -xCC" --without-readline --enable-dtrace DTRACEFLAGS="-64"

And with gmake I am ready to use the latest build of PostgreSQL on OpenSolaris.

3 comments:

Joe said...

It would be nice if configure checked for bison/flex first, but the requirement is documented: http://www.postgresql.org/docs/8.4/static/install-requirements.html (bottom of the page).

Anonymous said...

Configure does check for Bison/Flex, the problem is that if you are building from a source tarball (like an alpha release, or at one time the nightly downloads), those contain the pre-generated files you would need bison/flex for, so it isn't a fatal problem if you don't have Bison/Flex.

On another note, once you've gone through the pain of setting this up once (per Jignesh's example), if you want to do regular builds I would recommend using PEG (http://github.com/gregs1104/peg/), which automates many of these steps for you. I've worked with Greg to help make the scripts Solaris compatible, so they should work ok for most people. (If you find bugs on Solaris, please report them to Greg or myself)

Unknown said...

Hi Jignesh,

Nice article, very usefull.

I've takken it to make some customizations to install PostgreSQL 9.0 on OpenIndiana and publish a post on my blog -in spanish-, here's the post:

http://sparcki.blogspot.com/2010/09/instalacion-de-postgresql-en.html

Also, i've used the configure options as Bjorn uses to make the "oficial" PostgreSQL package for Solaris

Thanks,
Urko