1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<link title="new" rel="stylesheet" href="http://www.gentoo.org/css/main.css" type="text/css">
<link REL="shortcut icon" HREF="http://www.gentoo.org/favicon.ico" TYPE="image/x-icon">
<link rel="search" type="application/opensearchdescription+xml" href="http://www.gentoo.org/search/www-gentoo-org.xml" title="Gentoo Website">
<link rel="search" type="application/opensearchdescription+xml" href="http://www.gentoo.org/search/forums-gentoo-org.xml" title="Gentoo Forums">
<link rel="search" type="application/opensearchdescription+xml" href="http://www.gentoo.org/search/bugs-gentoo-org.xml" title="Gentoo Bugzilla">
<link rel="search" type="application/opensearchdescription+xml" href="http://www.gentoo.org/search/packages-gentoo-org.xml" title="Gentoo Packages">
<link rel="search" type="application/opensearchdescription+xml" href="http://www.gentoo.org/search/archives-gentoo-org.xml" title="Gentoo List Archives">
<title>Gentoo Linux Documentation
--
Position Independent Code internals</title>
</head>
<body style="margin:0px;" bgcolor="#ffffff"><table width="100%" border="0" cellspacing="0" cellpadding="0">
<tr><td valign="top" height="125" bgcolor="#45347b"><a href="http://www.gentoo.org/"><img border="0" src="http://www.gentoo.org/images/gtop-www.jpg" alt="Gentoo Logo"></a></td></tr>
<tr><td valign="top" align="right" colspan="1" bgcolor="#ffffff"><table border="0" cellspacing="0" cellpadding="0" width="100%"><tr>
<td width="99%" class="content" valign="top" align="left">
<br><h1>Position Independent Code internals</h1>
<form name="contents" action="http://www.gentoo.org">
<b>Content</b>:
<select name="url" size="1" OnChange="location.href=form.url.options[form.url.selectedIndex].value" style="font-family:sans-serif,Arial,Helvetica"><option value="#doc_chap1">1. Excerpt</option>
<option value="#doc_chap2">2. Introduction to PIC - (Position Independent Code)</option>
<option value="#doc_chap3">3. Using PIC for building shared libraries</option>
<option value="#doc_chap4">4. Relocations in the TEXT segment of shared libraries used by dynamically linked executables</option>
<option value="#doc_chap5">5. Using prelink and LD_BIND_NOW</option>
<option value="#doc_chap6">6. More about negative side effects of text relocations in shared libraries</option>
<option value="#doc_chap7">7. So, why not use -fPIC building as default for all applications?</option>
<option value="#doc_chap8">8. Conclusion</option>
<option value="#doc_chap9">9. References</option></select>
</form>
<p class="chaphead"><a name="doc_chap1"></a><span class="chapnum">1.
</span>Excerpt</p>
<p>
This technical documentation tries to explain and evaluate the technical background and the performance benefit or likewise penalty of PIC (Position Independent Code).
</p>
<p>
The goal should be achieved by illustrating an easy to follow learning path to understand text relocations and why they are imposing a security risk and a speed penalty to running applications.
</p>
<p>
To enhance the reading comfort for beginners, it is not covering stack layouts, the technical details of starting functions and discussing internal toolchain processings during building and executing programs.
We are aware of the fact that this document may put a smile on the face of experienced readers due to the sometimes barely justified oversimplification of technical internals.
</p>
<p class="chaphead"><a name="doc_chap2"></a><span class="chapnum">2.
</span>Introduction to PIC - (Position Independent Code)</p>
<p>
PIC code differs from traditional code in the method it will perform access to function code and data objects/variables through an indirect accessing table.
This table is called the "Global Offset Table" because it contains the addresses of code functions and data objects exported by a shared library.
</p>
<p>
The dynamic loader modifies the GOT slots to resemble the current memory address for every exported symbol in the library.
When the dynamic loader has completed, the GOT contains full absolute addresses for each symbol reference constructed from the load address (PT_LOAD) of the shared library that contains these symbols plus their offset inside this shared library.
</p>
<p class="chaphead"><a name="doc_chap3"></a><span class="chapnum">3.
</span>Using PIC for building shared libraries</p>
<p>
Besides for using position independent executables (see PIE-SSP docs, PaX specs files using -shared and Jelinek binutils patches for -pie support),
the natural reason for using "-fPIC" (position independent code) is the use in shared dynamic libraries.
This makes the overall footprint of all dynamically linked ELF executables on the system as small as possible,
while it also prevents possible code duplication and actively reduces requirements on memory and file system.
</p>
<p>
A unique characteristic of a typical shared library is that it can be located anywhere in the process memory layout.
Because of this, the contents of the shared library are not accessed directly but via clearly exported definitions in symbol tables during building.
Today, shared libraries can be easily implemented by incorporating this key advantage of the ELF standard.
Making libraries larger, smaller or moving around functions in the library is very easy as long as the symbol table to access the functions does not change.
</p>
<p>
During building, The linker is only responsible for setting up exported symbols of the library in question.
Telling the object code that it needs to be position independent is the task of the preprocessor and the compiler.
Here, the role of the Makefiles and the CFLAGS/LDFLAGS feeding the compiler with instructions becomes visible.
The preprocessor is adding special definitions ("__PIC__" "__pic__") and the compiler is using "-fPIC" or "-fpic" depending on the data access model.
Hopefully, when there is no PIC unaware assembler in the source code, these flags are generating the object code needed for position independence.
The object code needs to be generated PIC for successfully opening the doors to position independent relocation of the library, created from the PIC .o relocatable objects.
</p>
<p class="chaphead"><a name="doc_chap4"></a><span class="chapnum">4.
</span>Relocations in the TEXT segment of shared libraries used by dynamically linked executables</p>
<p>
This chapter is going to explain the reasons why relocations in the TEXT segment of a library, also called "text relocations", must be avoided by designers of shared libraries.
</p>
<p>
The performance penalty of text relocations is the reason that every shared library object code should be generated with -fPIC or -fpic, depending on the addressing range of the data that is used.
Otherwise the library is considered not "clean".
</p>
<p>
A text relocation is a memory address in the "LOAD READ-EXECUTE" text segment of a shared library where text segment means the segment that contains the program code.
Such a nonPIC text segment often contains large amounts of memory addresses that need to be "patched" (manipulated, modified, corrected) with the runtime location of functions and data.
This is performed by the dynamic loader (ld.so in glibc) during startup of the dynamically linked executable and invocation of these libraries in the process space.
The reason that the dynamic loader needs to spend so much time "patching" memory addresses (relocations) was stated above:
a unique characteristic of a typical shared library is that it can be located anywhere in the process memory layout.
</p>
<p>
So the dynamic loader is the key to the "located anywhere" functionality: it recognizes and reorganizes the memory addresses that need to be refurbished and applies the change to these locations.
This means that the dynamic loader will be responsible for relocating the memory address.
For example, in a non-PIC compiled libmpeg3 library there are roughly 6000 memory locations left inside the shared library to point to some 200-300 functions and data referred by the instructions.
</p>
<p class="chaphead"><a name="doc_chap5"></a><span class="chapnum">5.
</span>Using prelink and LD_BIND_NOW</p>
<p>
Using prelink somehow mitigates the performance-intensive relocation process to a one-time operation: the relocation is satisfied and prematurely resolved/patched inside the binaries and the nonPIC shared libraries.
This can be reached with using a program like prelink that is working on the actual files and modifying the relocations and GOT slots in the executables and libraries directly, thus saving the dynamic loader a lot of work during actually starting and running the executable.
While dealing with executables, note that prelink inserts a "hint" into the PT_LOAD segment of every shared library to make the kernel load it at the expected address.
</p>
<p>
Bear in mind that not all relocations are resolved at startup time.
When LD_BIND_NOW is not used, the lazy binding for libraries somehow tries to minimize the overhead to a more timely fashion by only relocating symbols at their first invocation during program flow.
The environment variable LD_BIND_NOW (and the ld switch "-z now") tries to address this problem for slow machines by moving all needed relocations to be done at startup of a binary, invoking much much slower startup times but later making the binary run more fluently on slower machines because relocations are satisfied now :-)
But you should be careful and know that using LD_BIND_NOW is not recommended on machines where responsiveness is an issue, clicking on an icon in KDE or GNOME and waiting 20 seconds for evolution to start is sometimes inacceptable by users.
In doubt, use prelink!
</p>
<p class="chaphead"><a name="doc_chap6"></a><span class="chapnum">6.
</span>More about negative side effects of text relocations in shared libraries</p>
<p>
There are two drawbacks of nonPIC shared libraries currently.
</p>
<p>
There is a moderate security risk on nonPIC libraries containing text relocations.
The TEXT pages for shared libraries cannot be marked READONLY by the kernel starting the binary and mapping in ELF segments and libraries.
</p>
<p>
There is also a memory overhead and performance penalty: data and code cannot be shared amongst processes via COW.
COW means "copy on write", this uses the same readonly memory pages for all instances of the same binary and for all processes referring to the same used libraries.
Using COW, readonly memory pages for shared libraries and executables need not to be recreated for new processes in their memory until they are are about to be changed by the new process, like for patching in text relocations by the dynamic loader, this is all implemented transparently in the virtual memory management of the Linux kernel.
</p>
<p class="chaphead"><a name="doc_chap7"></a><span class="chapnum">7.
</span>So, why not use -fPIC building as default for all applications?</p>
<p>
So why not compile all applications with -fPIC if it has so much advantages?
The impact of "-fPIC" on certain arches like AMD64 can be tolerated due to the true PIC-oriented data and code addressing scheme and is even necessary on several (considered broken) architectures that refuse to build certain applications without -fPIC (errors with nonPIC relocation types on PARISC).
There is only one official method to add flags like "-fPIC" to ebuilds: using the flag-o-matic eclass and "append-flags".
However, it is not a good idea to enable "-fPIC" in global CFLAGS or create ebuilds that automatically add the "-fPIC" flag independent of the situation and architecture it is applied.
There are people referring to a noticeable performance penalty when running executables containing position independent code compared to executables incorporating normally compiled object code.
</p>
<p>
Normally, the setup of the PIC register takes about three assembler commands per function that is entered and additional overhead of 1-2 assembler commands per accessed symbol (code function or data object).
Thus, we have the inverted situation for normal executables that the invocation of the "-fPIC" flag is doing the exact opposite like in shared libraries.
Instead of giving us speed for low memory profile by saving memory via COW and making text relocations unnecessary, the additional overhead in the addressing mode is imposing a speed penalty on our executable.
</p>
<p>
Why does a normal dynamically linked executable (not position independent shared executable) need no text relocations and PIC addressing?
Because the kernel (in a normal world) always moves it to the same location in process memory when started, making it unnecessary for the dynamic loader to address any TEXT relocations in the normal executable: because there are none!
We have learned that only shared libraries are located at a given, freely choosen, address space in the process memory of the dynamically linked executable.
So, in the text segment of a "fixed load location" normal executable there are no TEXT segment relocations because all addresses are at the same location in memory during every invocation of the program.
The addressing of data and functions inside the executable are provided via relative and absolute relocations in a common used set of platform-dependent, performance oriented assembler commands.
</p>
<p>
While the architectures supported by Gentoo are quite differently addressing memory, they all share the same characteristic: direct non-PIC-aware addressing is always cheaper (read: faster) than PIC addressing.
For example the RISC (Reduced Instruction Set) architectures sparc, ppc and hppa sometimes use more than one assembler command issuing several more opcodes to do what x86 does with a single variable length assembler command, loading a full 32-Bit address for example.
Only the AMD64 seems to support some kind of "emulation" mode where it does not seem to make a difference if PIC or normal addressing is used for referring code functions and data to access.
</p>
<p class="chaphead"><a name="doc_chap8"></a><span class="chapnum">8.
</span>Conclusion</p>
<p>
The only way that time-wasting text relocations are imposed on a process, leading to the dynamic loader having to work overtime, are with using nonPIC dynamically shared libraries.
</p>
<p>
For normal executables that are dynamically linked to these shared libraries, the executables themselves need not to be using -fPIC for building the object code they consist of.
These executables simply do not need the PIC addressing mode for their functions and data and will use the PLT (Process Linkage Table) and the GOT (Global Offset Table) anyway for addressing external data in shared libraries.
</p>
<p class="chaphead"><a name="doc_chap9"></a><span class="chapnum">9.
</span>References</p>
<ul>
<li><a href="pic-guide.xml">Introduction to Position Independent Code</a></li>
<li>
<a href="http://www.iecc.com/linker/">Linkers and Loaders</a> by Levine (the Levine book)</li>
<li>
<a href="http://people.redhat.com/drepper/dsohowto.pdf">How to Write Shared Libraries</a> by Ulrich Drepper</li>
</ul>
<p>I would like to say personal thanks to the PaX team for supporting us with an extraordinary and outstanding commitment to our toolchain issues!</p>
<br><p class="copyright">
The contents of this document, unless otherwise expressly stated, are licensed under the <a href="http://creativecommons.org/licenses/by-sa/2.5">CC-BY-SA-2.5</a> license. The <a href="http://www.gentoo.org/main/en/name-logo.xml"> Gentoo Name and Logo Usage Guidelines </a> apply.
</p>
<!--
<rdf:RDF xmlns="http://web.resource.org/cc/"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<license rdf:about="http://creativecommons.org/licenses/by-sa/2.5/">
<permits rdf:resource="http://web.resource.org/cc/Reproduction" />
<permits rdf:resource="http://web.resource.org/cc/Distribution" />
<requires rdf:resource="http://web.resource.org/cc/Notice" />
<requires rdf:resource="http://web.resource.org/cc/Attribution" />
<permits rdf:resource="http://web.resource.org/cc/DerivativeWorks" />
<requires rdf:resource="http://web.resource.org/cc/ShareAlike" />
</License>
</rdf:RDF>
--><br>
</td>
<td width="1%" bgcolor="#dddaec" valign="top"><table border="0" cellspacing="4px" cellpadding="4px">
<tr><td class="topsep" align="center"><p class="altmenu"><a title="View a printer-friendly version" class="altlink" href="pic-internals.xml?style=printable">Print</a></p></td></tr>
<tr><td class="topsep" align="center"><p class="alttext">Page updated February 14, 2004</p></td></tr>
<tr><td class="topsep" align="left"><p class="alttext"><b>Summary: </b>Understanding the impact of text relocations and explaining the use of PIC in shared libraries</p></td></tr>
<tr><td align="left" class="topsep"><p class="alttext">
<a href="mailto:a.gabert@fh-trier.de" class="altlink"><b>Alexander Gabert</b></a>
<br><i>Author</i><br><br>
<a href="mailto:solar@gentoo.org" class="altlink"><b>solar</b></a>
<br><i>Contributor</i><br><br>
<a href="mailto:pageexec@freemail.hu" class="altlink"><b>The PaX team</b></a>
<br><i>Contributor</i><br></p></td></tr>
<tr lang="en"><td align="center" class="topsep">
<p class="alttext"><b>Donate</b> to support our development efforts.
</p>
<form action="https://www.paypal.com/cgi-bin/webscr" method="post">
<input type="hidden" name="cmd" value="_xclick"><input type="hidden" name="business" value="paypal@gentoo.org"><input type="hidden" name="item_name" value="Gentoo Linux Support"><input type="hidden" name="item_number" value="1000"><input type="hidden" name="image_url" value="http://www.gentoo.org/images/paypal.png"><input type="hidden" name="no_shipping" value="1"><input type="hidden" name="return" value="http://www.gentoo.org"><input type="hidden" name="cancel_return" value="http://www.gentoo.org"><input type="image" src="http://images.paypal.com/images/x-click-but21.gif" name="submit" alt="Donate to Gentoo">
</form>
</td></tr>
<tr lang="en"><td align="center"><iframe src="http://sidebar.gentoo.org" scrolling="no" width="125" height="850" frameborder="0" style="border:0px padding:0x" marginwidth="0" marginheight="0"><p>Your browser does not support iframes.</p></iframe></td></tr>
</table></td>
</tr></table></td></tr>
<tr><td colspan="2" align="right" class="infohead">
Copyright 2001-2012 Gentoo Foundation, Inc. Questions, Comments? <a class="highlight" href="http://www.gentoo.org/main/en/contact.xml">Contact us</a>.
</td></tr>
</table></body>
</html>
|
GitWeb