Linear Algebra on Gentoo

- -1.0 -2010-12-22 - - -Introduction -

- - -

- There are many - performant numerical libraries available. - The Basic Linear Algebra Subprograms (BLAS) and the Linear Algebra PACKage (LAPACK) - are well designed linear algebra libraries developed by the - High Performance Computing (HPC) community. BLAS is an API of dense - matrix and vectors products, while LAPACK provides routines for - solving systems of linear equations. Both are widely used in - many scientific applications and it is, therefore, important to - have efficient implementations available. -

- -

- BLAS and LAPACK were originally written in FORTRAN 77. Since then, a - number of additional language wrappers have been developed for - languages like C, C++, FORTRAN 95, Java, Python, etc... - Netlib offers exact implementations of the APIs and they are called - "reference" libraries. There is also some parallel implementations - for -

- -

- BLAS: FORTRAN 77 and C - (CBLAS) implementations of BLAS -
- LAPACK: FORTRAN 77 and - C (LAPACKE) implementations of LAPACK -

- -

- BLACS: FORTRAN 77 and C - implementations of BLACS -
- ScaLAPACK: FORTRAN 77 and - C implementations of PBLAS and ScaLAPACK -

- -

- In addition, Gentoo provides a number of optimized implementations - of the above linear algebra libraries that will be described - below. You can switch between implementations with the - Gentoo's eselect system and the widely used pkg-config - tool. -

- -

- It is important to note that if you require, e.g., a well performing - BLAS implementation, simply emerging X over Y often is not enough. Rather, you will have - to carefully benchmark your applications since performance may depend - on many factors, - such as hardware or network. - If you are simply looking for a well performing and well tested - implementation, the reference ebuilds will likely be your best choice. -

- - - -

- - - -For Users -

-Installing - - -

- If best possible performance is not of paramount importance for you - and you simply need BLAS and/or LAPACK, just emerge the virtual - package: -

- -

-# emerge lapack
-

- -

- This will install both <><> and <><> the reference packages from - http://www.netlib.org/ . They are well tested, easy to debug - implementations. They should satisfy most users; if they're all you need, you're - done reading. -

- -

-However, if: -

- -

linear algebra libraries are critical for the speed of your applications
you absolutely need to build the fastest computer
you want to help Gentoo sci project to improve their packages

- -

-... then read on, and be sure to file bugs both to Gentoo and upstream. -

- -

- There is a number of optimized implementations of these libraries in the Portage - tree: -

- -

- ATLAS: Automatically - Tuned Linear Algebra Software is an open-source package that empirically - tunes the library to the machine it is being compiled on. It provides BLAS - (FORTRAN 77 and C), and LAPACK implementations on various architectures. -
- GotoBLAS: - Goto BLAS provides open-source, free for academic use, hand-coded - machine language, processor optimized versions of the FORTRAN 77 - and C BLAS routines. Still claims to be the fastest BLAS. -
- ACML: - AMD Core Math Library is a closed-source but free package containing BLAS (FORTRAN 77 - only) and LAPACK for x86 and x86_64 architectures, but also other math tools - such as statistical libraries and FFTs. -
- MKL: - Intel® Math Kernel Library is a closed-source but free package for - non-commercial use on Linux systems containing implementations of all the linear - algebra libraries mentioned here. -

- -

- Usually performance gain is noticeable mainly with BLAS, since LAPACK routines - depend on BLAS kernels. -

- - -

-Developping with the installed linear algebra libraries - - -

- We took great care to make sure that each package provides - consistent pkg-config files generated by us. - Compiling and linking becomes straightforward: -

- -

-# pkg-config --libs blas    (To link with FORTRAN 77 BLAS library)
-# pkg-config --cflags cblas (To compile against C BLAS library)
-# pkg-config --libs cblas   (To link with C BLAS library)
-# pkg-config --libs scalapack  (To link with the ScaLAPACK library)
-

- -

- pkg-config files are available for all implementations and - various alternatives within implementations. The default names of - the implementations are: blas, cblas, lapack, lapacke, blacs and - scalapack, and they can be chosen with eselect. You can also always compile or link - with an library not selected for the - More information on using pkg-config can be obtained with man pkg-config. -

- - -

-Selecting libraries - - -

- You can switch BLAS, CBLAS and LAPACK implementations with - eselect. you can view which implementations of CBLAS - are available. -

- -

-# eselect cblas list
-Installed CBLAS for library directory lib64
-[1]   atlas
-[2]   atlas-threads
-[3]   gsl
-[4]   mkl-threads *
-[5]   reference
-

- -

- The implementation marked with an asterisk (*) is the currently - selected implementation. To switch implementations, run: -

- -

-# eselect blas set atlas-threads
-

- -

- To learn more about the eselect tool, visit the - eselect guide -

- -

- When selecting your linear algebra profiles try to avoid mixing - different implementations since we don't have any mechanism to enforce - reasonable profiles. However, here is a list of well performing - profile combinations that have been used successfully in the past: -

performant on most CPUs: -
- blas, cblas: atlas (or atlas-threads with multi-processor)
- lapack, lapacke: atlas
-
performant on most CPUs: -
- blas, cblas: goto2
- lapack, lapacke: reference
-
performant on AMD based CPUs: -
- blas, lapack: acml-gfortran (or acml-gfortran-openmp with - multi-processors)
- cblas: reference
-
performant on Intel based CPUs: -
- blas,cblas,lapack: mkl-threads
-

- - -

- -

-Choosing a compiler - - -

- All the above libraries have been tested with the GNU compiler - collections (gcc, gfortran). - There are many available C compilers and a few FORTRAN (ifort, - Open64) compilers on Gentoo and many other FORTRAN compilers outside - of Gentoo (). -

- -

-# F77=ifort FFLAGS="-O2 -mp1" emerge blas-reference
-

- -

- Depending on your hardware, a small performance gain can be noticed thanks to - vectorization. The -mp flag maintains floating-point precision, since by - default ifort is pretty aggressive on floating point arithmetic, and we are - actually compiling a math package. Try man ifort to see additional flags - to fit your hardware. -

- -

- Some of the implementations let you specify the Intel® C compiler as - well. Please beware that not all libraries compile with all - combinations. You should receive an error during the emerge in case you have - chosen an incompatible combination. -

- -

- As usual for Gentoo, there are many combinations of USE flags and - compilers with which you could compile a package. Unfortunately - switching compilers between BLAS and LAPACK might not be always - compatible. For example: -

- -

-# USE=ifort emerge acml
-# eselect blas set acml-ifort-openmp
-# FC=gfortran FFLAGS="-O2" emerge lapack-reference
-

- -

- This will most likely break things or not even compile. -

- -

- Try to be consistent in your choice. Stay with the GCC most of the time will - avoid you some trouble, unless you want to use the MKL, in which case the Intel - compilers make a good combination. -

- - -

-Documentation - - -

- If you need BLAS or LAPACK to develop your own programs, the documentation - becomes pretty handy. Setting the USE="doc" flag for the corresponding BLAS or - LAPACK package will install man pages and quick reference sheets from the - app-doc/blas-docs and app-doc/lapack-docs packages. They are - standard and valid for all implementations. For optimized packages, the - USE="doc" flags will usually install extra doc in PDF or HTML format. -

- - -

- - - -For ebuild developers -

- -

-Packages with BLAS/LAPACK dependencies - - -

- You need two things: - set [R]DEPEND to virtual/. To build some - packages, you m need to use the pkg-config tool. If you are lucky, the - package uses autotools together with the autoconf <>AX_BLAS and <>AX_LAPACK M4 - macros. In this case, the configuration step becomes simple. For example: -

- -

-econf --with-blas="$(pkg-config --libs blas)"
-

- - -

- - - -Providing new implementations - - -

- The Portage tree contains many ebuilds that depend on the - BLAS/CBLAS/LAPACK/BLACS/ScaLAPACK libraries. As there is more than - one possible implementation, the Gentoo Science Project - reorganized all the packages to provide virtual. All ebuilds using - should depend on this virtual package, unless it is explicitly - known to break with a specific implementation. -

- -

- To work with Gentoo's configuration tools - app-admin/eselect-{blas,cblas,lapack}, and the virtual, every ebuild that - installs a BLAS implementation must fulfill following requirements: -

- -

- The ebuild must install an eselect file for each profile it provides. The - libraries should link to the ones in /usr/$(get_libdir) - directories and the include files in /usr/include: -
- - libblas.so[.0] - Shared object for FORTRAN BLAS - applications -
- - libblas.a - Static library for FORTRAN BLAS applications -
- - libcblas.so[.0] - Shared object for C/C++ CBLAS applications -
- - libcblas.a - Static library for C/C++ CBLAS applications -
- cblas.h - Include header for C/C++ applications
- - liblapack.so[.0] - Shared object for FORTRAN LAPACK - applications -
- - liblapack.a - Static library for FORTRAN LAPACK applications -
-
- The ebuild must install a blas.pc, cblas.pc and/or - lapack.pc pkg-config file and therefore RDEPEND on - dev-util/pkgconfig. They should also be included in the eselect - files, and link to the /usr/$(get_libdir)/pkgconfig directory: -
- blas.pc - BLAS pkg-config file
- cblas.pc - CBLAS pkg-config file
- lapack.pc - LAPACK pkg-config file
-
Be included in the virtual package as a possible provider: -
- virtual/blas - BLAS virtual package
- virtual/cblas - CBLAS virtual package
- virtual/lapack - LAPACK virtual package
-

- -

- The easiest way of understanding all this is probably getting inspiration from - one of the available packages. Currently the Portage tree provide the following - virtual packages: -

- - - - - - - - - - - - - sci-libs/acml - * - - * - - - - - - sci-libs/atlas - * - * - * - * - - - - - sci-libs/gotoblas2 - * - * - - - - - - - sci-libs/blas-reference - * - - - - - - - - sci-libs/cblas-reference - - * - - - - - - - sci-libs/gsl - - * - - - - - - - sci-libs/lapack-reference - - - * - - - - - - sci-libs/mkl - * - * - * - * - * - * - -

Package name	virtual/blas	virtual/cblas	virtual/lapack	virtual/lapacke	virtual/blacs	virtual/scalapack

- - -

- - - - -Benchmarks -

- - -

- If you feel inclined to write an ebuild for these, you - are more than welcomed to file it on our Bugzilla. -

- - -