diff options
author | Diego Elio 'Flameeyes' Pettenò <flameeyes@gmail.com> | 2010-01-06 22:57:19 +0100 |
---|---|---|
committer | Diego Elio 'Flameeyes' Pettenò <flameeyes@gmail.com> | 2010-01-06 22:57:19 +0100 |
commit | ea020141aa6f0b3b1ab3c17d0d2da0633f19aa46 (patch) | |
tree | a7168cd14bde5a35c22468af26da4817bbdf3a15 | |
parent | Reduce General Concepts' chapters to sections. (diff) | |
download | devmanual-ea020141aa6f0b3b1ab3c17d0d2da0633f19aa46.tar.gz devmanual-ea020141aa6f0b3b1ab3c17d0d2da0633f19aa46.tar.bz2 devmanual-ea020141aa6f0b3b1ab3c17d0d2da0633f19aa46.zip |
More conversions, finished chapter one.
-rw-r--r-- | chunk.toc | 4 | ||||
-rw-r--r-- | content/general-concepts.xmli | 4 | ||||
-rw-r--r-- | content/general-concepts/tree.xmli | 261 | ||||
-rw-r--r-- | content/general-concepts/use-flags.xmli | 210 | ||||
-rw-r--r-- | content/general-concepts/user-environment.xmli | 88 | ||||
-rw-r--r-- | content/general-concepts/virtuals.xmli | 100 |
6 files changed, 667 insertions, 0 deletions
@@ -22,6 +22,10 @@ <d:tocentry linkend="general-concepts.portage-cache"><?dbhtml filename="general-concepts/portage-cache/index.html"?></d:tocentry> <d:tocentry linkend="general-concepts.sandbox"><?dbhtml filename="general-concepts/sandbox/index.html"?></d:tocentry> <d:tocentry linkend="general-concepts.slotting"><?dbhtml filename="general-concepts/slotting/index.html"?></d:tocentry> + <d:tocentry linkend="general-concepts.tree"><?dbhtml filename="general-concepts/tree/index.html"?></d:tocentry> + <d:tocentry linkend="general-concepts.use-flags"><?dbhtml filename="general-concepts/use-flags/index.html"?></d:tocentry> + <d:tocentry linkend="general-concepts.user-environment"><?dbhtml filename="general-concepts/user-environment/index.html"?></d:tocentry> + <d:tocentry linkend="general-concepts.virtuals"><?dbhtml filename="general-concepts/virtuals/index.html"?></d:tocentry> </d:tocentry> </toc> diff --git a/content/general-concepts.xmli b/content/general-concepts.xmli index dc5172c..94a37e9 100644 --- a/content/general-concepts.xmli +++ b/content/general-concepts.xmli @@ -30,5 +30,9 @@ <xi:include parse="xml" href="general-concepts/portage-cache.xmli" /> <xi:include parse="xml" href="general-concepts/sandbox.xmli" /> <xi:include parse="xml" href="general-concepts/slotting.xmli" /> + <xi:include parse="xml" href="general-concepts/tree.xmli" /> + <xi:include parse="xml" href="general-concepts/use-flags.xmli" /> + <xi:include parse="xml" href="general-concepts/user-environment.xmli" /> + <xi:include parse="xml" href="general-concepts/virtuals.xmli" /> </chapter>
\ No newline at end of file diff --git a/content/general-concepts/tree.xmli b/content/general-concepts/tree.xmli new file mode 100644 index 0000000..0d2ff18 --- /dev/null +++ b/content/general-concepts/tree.xmli @@ -0,0 +1,261 @@ +<?xml version="1.0" encoding="utf-8"?> +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xl="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="general-concepts.tree"> + <title>The Portage Tree</title> + + <para> + The basic layout of the portage tree is as follows: + </para> + + <itemizedlist> + <listitem> + <para> + Categories, for example <filename>app-editors</filename>, <filename>sys-apps</filename> + </para> + + <itemizedlist> + <listitem> + <para> + Category metadata, for example <filename>app-admin/metadata.xml</filename> + </para> + </listitem> + + <listitem> + <para> + Package directories for example <filename>app-editors/vim</filename> + </para> + + <itemizedlist> + <listitem> + <para> + Package metadata, for example <filename>app-editors/vim/metadata.xml</filename> + </para> + </listitem> + + <listitem> + <para> + Package changelog, for example <filename>app-editors/vim/ChangeLog</filename> + </para> + </listitem> + + <listitem> + <para> + Package Manifest, for example <filename>app-editors/vim/Manifest</filename> + </para> + </listitem> + + <listitem> + <para> + Ebuilds, for example <filename>app-editors/vim/vim-6.3.068.ebuild</filename>, + <filename>app-editors/vim/vim-7.0_alpha20050326.ebuild</filename> + </para> + </listitem> + + <listitem> + <para> + Package files directory, for example <filename>app-editors/vim/files</filename> + </para> + + <itemizedlist> + <listitem> + <para> + Small patches and other miscellaneous files, for example + <filename>app-editors/vim/files/vim-completion</filename> + </para> + </listitem> + </itemizedlist> + </listitem> + </itemizedlist> + </listitem> + </itemizedlist> + </listitem> + + <listitem> + <para> + Eclasses directory, <filename>eclass/</filename> + </para> + </listitem> + + <listitem> + <para> + Licenses directory, <filename>licenses/</filename> + </para> + </listitem> + + <listitem> + <para> + Profiles directory, <filename>profiles/</filename> + </para> + + <itemizedlist> + <listitem> + <para> + Various control and documentation files, for example <filename>categories</filename>, + <filename>info_pkgs</filename>, <filename>info_vars</filename>, <filename>package.mask</filename>, + <filename>use.desc</filename> + </para> + </listitem> + + <listitem> + <para> + Updates directory, <filename>profiles/updates/</filename> + </para> + + <itemizedlist> + <listitem> + <para> + Updates files, for example <filename>profiles/updates/1Q-2005</filename> + </para> + </listitem> + </itemizedlist> + </listitem> + + <listitem> + <para> + Main profile cascade + </para> + </listitem> + </itemizedlist> + </listitem> + + <listitem> + <para> + Scripts directory, <filename>scripts/</filename> + </para> + </listitem> + + <listitem> + <para> + Distfiles directory, <filename>distfiles/</filename>. This is not included in the main CVS tree, but it will be + found on most user systems. + </para> + </listitem> + + <listitem> + <para> + Packages directory, <filename>packages</filename>. Again, this is found on user systems but not in the main CVS + tree. + </para> + </listitem> + </itemizedlist> + + + <section> + <title>What Belongs in the Tree?</title> + + <para> + Things that do <emphasis>not</emphasis> belong in the tree: + </para> + + <itemizedlist> + <listitem><para>Large patches</para></listitem> + <listitem><para>Non-text files</para></listitem> + <listitem><para>Photos of teletubbies</para></listitem> + <listitem><para>Files whose name starts with a dot</para></listitem> + </itemizedlist> + + <para> + Software-wise, in general all of the following should be met in order for a package to be included in the tree: + </para> + + <variablelist> + <varlistentry> + <term>Active, Cooperative Upstream</term> + <listitem> + <para> + If a package is undeveloped or unmaintained upstream, it can be extremely difficult to get problems + fixed. If a package does not have an active upstream, the developers who add the package to the tree must + ensure that they are able to fix any issues which may arise. + </para> + + <para> + Sometimes upstream may have a reason for not wanting their package included in the tree. This should be + respected. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Reasonably Stable</term> + <listitem> + <para> + Keep super-experimental things out of the tree. If you must commit them, consider using + <filename>package.mask</filename> until things calm down, or better yet make them available as overlay + ebuilds. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Reasonably Useful</term> + <listitem> + <para> + Don't feel obliged to include "Joe's '1337 XMMS Skinz Collection" or "Hans' Super Cool Fast File System" in + the tree just because a few users ask for it. Stick to things that might actually be of use. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Properly Packaged</term> + <listitem> + <para> + If something is only available in live CVS or dodgy autopackage format, don't include it until upstream can + come up with a decent source package. Similarly, avoid things that don't have a proper build system (where + relevant) — these are very tricky to maintain. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Patching and Distribution Permitted</term> + <listitem> + <para> + If we can't patch packages as necessary ourselves, we end up relying entirely upon upstream for + support. This can be problematic, especially if upstream are slow at fixing things. We don't want to be in + the situation where we can't stable a critical package because we're still waiting for a closed-source + vendor to get their act together. + </para> + + <para> + Similarly, not being able to mirror and distribute tarballs ourselves makes us rely entirely upon upstream + mirrors. Experience has shown that these are often extremely unreliable, with files changing, moving or + vanishing at random. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Working Ebuilds</term> + <listitem> + <para> + If you don't have a <emphasis>working</emphasis> ebuild, don't include it. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Portable</term> + <listitem> + <para> + If software is unportable, it's generally because it's badly written. Remember that although x86 has a + market majority <emphasis>now</emphasis>, it probably won't in the not too distant future once x86-64 + catches on. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Reasonable Security Record</term> + <listitem> + <para> + Don't include software that has a terrible security record. Each vulnerability is a <emphasis>lot</emphasis> + of work for a lot of people (security teams, arch teams and package maintainers). + </para> + </listitem> + </varlistentry> + </variablelist> + </section> +</section> diff --git a/content/general-concepts/use-flags.xmli b/content/general-concepts/use-flags.xmli new file mode 100644 index 0000000..5a30627 --- /dev/null +++ b/content/general-concepts/use-flags.xmli @@ -0,0 +1,210 @@ +<?xml version="1.0" encoding="utf-8"?> +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xl="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="general-concepts.use-flags"> + <title>USE Flags</title> + + <para> + <varname>USE</varname> flags are to control optional dependencies and settings which the user may reasonably want to + select. For example, <package>vim</package> can optionally build with support for the Ruby language interpreter, and + it needs <package>ruby</package> installed to do this — we use the ruby <varname>USE</varname> flag to provide this + option. On the other hand, <package>glark</package> requires <package>ruby</package> no matter what, so no + <varname>USE</varname> flag is used there. + </para> + + <para> + No combination of <varname>USE</varname> flags should cause a package to fail to build. + </para> + + <para> + Packages should not configure and link based upon what is available at compile time — any autodetection must be + overridden. + </para> + + <note> + <para> + The status of <varname>USE</varname> flags is saved in the VDB, and their value in <function>pkg_prerm</function> and + <function>pkg_postrm</function> is taken from there. This means that setting or unsetting a USE flag between merge + and unmerge has no effect. + </para> + </note> + + + <section> + <title><literal>noblah</literal> <varname>USE</varname> Flags</title> + + <para> + If at all possible, avoid <literal>noblah</literal> style <varname>USE</varname> flags. These break + <filename>use.mask</filename> and cause all sorts of complications for arch developers. Here's why: + </para> + + <para> + Consider a hypothetical package named <package>vplayer</package>, which plays videos. This package has optional + support, via <varname>USE</varname> flags, for various sound and video output methods, various video codecs and so + on. + </para> + + <para> + One of vplayer's optional features is support for the 'fakemedia' codec, which is unfortunately only available as + a dodgy x86 binary. We <emphasis>could</emphasis> handle this by doing something like: + </para> + + <programlisting language="ebuild"><![CDATA[ +RDEPEND="x86? ( fakemedia? ( >=media-libs/fakemedia-1.1 ) )" +]]></programlisting> + + <para> + Except this is pretty nasty — what happens when an AMD64 binary is made as well? Also, users on other archs will + see fakemedia listed in <command>emerge -pv</command> output, even though it is not actually available. + </para> + + <para> + Similarly, say vplayer supports output via the ALSA codec as one option. However, ALSA isn't (or wasn't when this + example was written) available on SPARC or Alpha. So we could do: + </para> + + <programlisting language="ebuild"><![CDATA[ +DEPEND="!sparc? ( !alpha? ( alsa? ( media-libs/alsa-lib ) ) )" +]]></programlisting> + + <para> + Again, it's messy, and ALSA still shows up in the <command>emerge -p</command> output. Also, once ALSA starts + working on SPARC, every ebuild that does this would have to be manually edited. + </para> + + <para> + The solution is <filename>use.mask</filename>, which is documented in <xref linkend="profiles.use.mask."/>. Each + profile can have a <filename>use.mask</filename> file which can be used to forcibly disable certain USE flags on a + given arch (or subarch, or subprofile). So, if the <literal>fakemedia</literal> <varname>USE</varname> flag was + use.masked on every non-x86 profile, the following would be totally legal and wouldn't break anything: + </para> + + <programlisting language="ebuild"><![CDATA[ +RDEPEND="fakemedia? ( >=media-libs/fakemedia-1-1 )" +]]></programlisting> + + <para> + Users of non-x86 would see the following when doing <command>emerge -pv vplayer</command>: + </para> + + <programlisting><![CDATA[ +[ebuild R ] media-video/vplayer-1.2 alsa -blah (-fakemedia) xyz +]]></programlisting> + + <para> + To get a flag added to <filename>use.mask</filename>, ask the relevant arch team. + </para> + </section> + + <section> + <title>Local and Global USE Flags</title> + + <para> + USE flags are categorised as either local or global. A global USE flag must satisfy several criteria: + </para> + + <itemizedlist> + <listitem><para>It is used by many different packages.</para></listitem> + <listitem><para>It has a general non-specific purpose.</para></listitem> + </itemizedlist> + + <para> + The second point is important. If the effect of the <literal>thing</literal> <varname>USE</varname> flag upon + <package>pkg-one</package> is substantially different from the effect it has upon <package>pkg-two</package>, + then <literal>thing</literal> is not a suitable candidate for being made a global flag. In particular, note that + if <literal>client</literal> and <literal>server</literal> USE flags are ever introduced, they can not be + global USE flags for this reason. + </para> + + <para> + Before introducing a new global USE flag, it must be discussed on the gentoo-dev mailing list. + </para> + </section> + + <section> + <title>USE Flag Descriptions</title> + + <para> + All USE flags must be described in either <filename>use.desc</filename> or <filename>use.local.desc</filename> in + the <filename>profiles/</filename> directory. See <command>man portage</command> or the comments in these files + for an explanation of the format. Remember to keep these files sorted. + </para> + + <para> + The exceptions to this are <varname>USE_EXPAND</varname> flags, which must be documented in the + <filename>profiles/desc/</filename> directory. One file per <varname>USE_EXPAND</varname> variable is required, + which must contain descriptions of the possible values this variable can take. See the comments in these files for + the format, and remember to keep them sorted. + </para> + </section> + + <section> + <title>Conflicting USE Flags</title> + + <para> + Occasionally, ebuilds will have conflicting USE flags for functionality. Checking for them and returning an error + is <emphasis>not</emphasis> a viable solution. Instead, you must pick one of the USE flags in conflict to favour. + </para> + + <para> + One example comes from the <package>msmtp</package> ebuilds. The package can use either SSL with GnuTLS, SSL + with OpenSSL, or no SSL at all. Because GnuTLS is more featureful than OpenSSL, it is favoured: + </para> + + <programlisting language="ebuild"><![CDATA[ +src_compile() { + local myconf + + if use ssl &&; use gnutls ; then + myconf="${myconf} --enable-ssl --with-ssl=gnutls" + elif use ssl &&; ! use gnutls ; then + myconf="${myconf} --enable-ssl --with-ssl=openssl" + else + myconf="${myconf} --disable-ssl" + fi + + econf \ + # Other stuff + ${myconf} + + emake || die "make failed" +} +]]></programlisting> + + </section> + + <section> + <title><varname>USE_EXPAND</varname> and <varname>ARCH</varname> <varname>USE</varname> Flags</title> + + + <para> + The <varname>VIDEO_CARDS</varname>, <varname>INPUT_DEVICES</varname> and <varname>LINGUAS</varname> variables are + automatically expanded into USE flags. These are known as <varname>USE_EXPAND</varname> variables. If the user has + <literal>LINGUAS="en fr"</literal> in <filename>make.conf</filename>, for example, then <literal>USE="linguas_en + linguas_fr"</literal> will automatically be set by Portage. + </para> + + <para> + The <varname>USE_EXPAND</varname> list is set in <filename>profiles/base/make.default</filename> as of Portage + 2.0.51.20. This must not be modified without discussion on the gentoo-dev list, and it must not be modified in any + subprofile. + </para> + + <para> + The current architecture (e.g. <literal>x86</literal>, <literal>sparc</literal>, <literal>ppc-macos</literal>) + will automatically be set as a USE flag as well. See <filename>profiles/arch.list</filename> for a full list of + valid architecture keywords, and <link xl:href="http://www.gentoo.org/proj/en/glep/glep-0022.html">GLEP 22</link> + for an explanation of the format. + </para> + + <warning> + <para> + It is a common misconception that the architecture variable is somehow related to + <varname>ACCEPT_KEYWORDS</varname>. It isn't. Accepting <literal>x86</literal> keywords on + <literal>sparc</literal>, for example, won't set <literal>USE="x86"</literal>. Similarly, there are no + <literal>~arch</literal> USE flags, so don't try <code>if use ~x86</code>. + </para> + </warning> + </section> +</section> diff --git a/content/general-concepts/user-environment.xmli b/content/general-concepts/user-environment.xmli new file mode 100644 index 0000000..bb714ec --- /dev/null +++ b/content/general-concepts/user-environment.xmli @@ -0,0 +1,88 @@ +<?xml version="1.0" encoding="utf-8"?> +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xl="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="general-concepts.user-environment"> + <title>User Environment</title> + + <para> + User environment variables and <filename>make.conf</filename> settings get passed on to ebuilds. This can be useful + — it's how <varname>CFLAGS</varname> and friends work, for example — but it can also result in nasty build-breaking + variables like <varname>LANG</varname> and <varname>LC_ALL</varname> getting through. Currently no sanitisation is + performed upon the environment. + </para> + + <section> + <title>Filtering Variables</title> + + <para> + Certain variables will really really upset certain build systems. A good example is the locale variables + (<varname>LC_ALL</varname> et al), which if set to certain values will cause <command>sed</command> or + <command>grep</command> expressions involving the likes of <literal>[A-Z]</literal> to fail. The easiest + thing to do here is to <command>unset</command> or sanitise the offending variables inside + <function>pkg_setup</function>. + </para> + + <para> + The simplest way to unset all locale-related variables is: + </para> + + <programlisting language="ebuild"><![CDATA[ +pkg_setup() { + # Unset all locale related variables, they can make the + # build fail. + + eval unset ${!LC_*} LANG +} +]]></programlisting> + </section> + + <section> + <title>Not Filtering Variables</title> + + <para> + On the other hand, it is extremely important that certain user preferences are honoured as far as possible. A good + example is <varname>CFLAGS</varname>, which <emphasis>must</emphasis> be respected (selective filtering is fine, + but outright ignoring is not). Ignoring <varname>CFLAGS</varname> when compiling can cause serious problems: + </para> + + <itemizedlist> + <listitem> + <para> + Ignoring <parameter>-march=/-mcpu=</parameter> may force kernel or software emulation for certain opcodes on + some architectures. This can be <emphasis>very</emphasis> slow — for example, <package>openssl</package> + built for SPARC v7 but run on v9 is around five times slower for RSA operations. + </para> + </listitem> + + <listitem> + <para> + Stripping certain ABI-related flags will break linkage. + </para> + </listitem> + + <listitem> + <para> + Stripping certain ABI-related flags will result in invalid code being produced for certain setups. In extreme + cases, we could end up with daft things like big endian code being produced for little endian CPUs. + </para> + </listitem> + + <listitem> + <para> + If a user's <parameter>-march=/-mcpu=/-mtune=</parameter> is ignored, and an auto-detected setting is used + instead, GRP and stages will break. For example, <literal>i686</literal> stages could no longer be produced on + a <literal>pentium-4</literal>, and <literal>v8</literal> stages could no longer be produced on an + <literal>UltraSparc</literal>. + </para> + </listitem> + </itemizedlist> + + <para> + Some packages do this by accident. For example, one might see <literal>CFLAGS=-Wall</literal> in + <filename>Makefile.am</filename>. To fix this, either <command>sed</command> in the user's + <varname>CFLAGS</varname>, or (the better solution) change the variable to <filename>AM_CFLAGS</filename>, which + will automatically be merged with the user's settings. <varname>LDFLAGS</varname> also should be respected. + </para> + </section> +</section> diff --git a/content/general-concepts/virtuals.xmli b/content/general-concepts/virtuals.xmli new file mode 100644 index 0000000..7611af6 --- /dev/null +++ b/content/general-concepts/virtuals.xmli @@ -0,0 +1,100 @@ +<?xml version="1.0" encoding="utf-8"?> +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xl="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="general-concepts.virtuals"> + <title>Virtuals</title> + + + <para> + Currently there exist two different forms of virtuals, our regular <varname>PROVIDE</varname> type virtuals, and so + called <emphasis>new-style</emphasis> virtuals. + </para> + + + <section> + <title><emphasis>Old-style</emphasis> Virtuals</title> + + <para> + <emphasis>Old-style</emphasis> virtuals are not really packages, but are something you can depend upon and + install. All <emphasis>old-style</emphasis> virtuals must have a category of "virtual". In order to use an + <emphasis>old-style</emphasis> virtual there are a few things that need to be present in the tree: + </para> + + <itemizedlist> + <listitem> + <para> + Atleast one ebuild must <varname>PROVIDE</varname> the virtual — see + <xref linkend="ebuild-writing.variables.optional-variables"/> + </para> + </listitem> + + <listitem> + <para> + An entry in the virtuals file for each profile to list the default provider + </para> + </listitem> + </itemizedlist> + + <para> + <emphasis>Old-style</emphasis> virtuals are not as flexible as <emphasis>new-style</emphasis> virtuals because + there is no concept of a version. You can only depend upon a virtual, but not a particular version of that + virtual. + </para> + + <para> + There are some things that are still only possible with old-style virtuals, + which is why they are still useful: + </para> + + <itemizedlist> + <listitem> + <para> + Packages providing the virtual can block the virtual, so you can ensure + nothing else is installed that also provides that virtual. + </para> + </listitem> + + <listitem> + <para> + The ability to set a default in the profile for what the virtual should + pull in (without the need of pulling the package into the system set). + </para> + </listitem> + </itemizedlist> + </section> + + <section> + <title><emphasis>New-style</emphasis> Virtuals</title> + + <para> + <emphasis>New-style</emphasis> virtuals are merely packages that are in the category of + <filename>virtual</filename>. They use their dependency string to specify the providers for the virtual and + should not install any files. Since they are regular ebuilds, there can be several versions of a virtual (which + can be helpful when a package may be provided by another in some versions, and not others — see the perl virtuals + for an example of this). One other difference (besides not installing any files) is that a + <emphasis>new-style</emphasis> virtual has an empty string for the value of the <varname>LICENSE</varname> and + <varname>HOMEPAGE</varname> variables. Since it installs no files, it really does not have a license. + </para> + + <para> + An example of a <emphasis>new-style</emphasis> virtual: + </para> + + <programlisting language="ebuild"><![CDATA[ +DESCRIPTION="Virtual for C++ tr1 <type_traits>" +HOMEPAGE="" +SRC_URI="" +LICENSE="" +SLOT="0" +KEYWORDS="alpha amd64 arm hppa ia64 mips ppc ppc64 s390 sparc x86 ~x86-fbsd" +IUSE="" +RDEPEND="|| ( >=sys-devel/gcc-4.1 dev-libs/boost )" +DEPEND="" +]]></programlisting> + + <para> + Looks familar...right? It should since its going to look just like a regular ebuild. + </para> + </section> +</section> |