libsandbox: use ptrace on apps that interpose their own allocator

If an app installs its own memory allocator by overriding the internal
glibc symbols, then we can easily hit a loop that cannot be broken: the
dlsym functions can attempt to allocate memory, and sandbox relies on
them to find the "real" functions.  So when someone calls a symbol that
the sandbox protects, we call dlsym, and that calls malloc, which calls
back into the app, and their allocator might use another symbol such as
open ... which is protected by the sandbox.  So we hit the loop like:
  -> open -> libsandbox:open -> dlsym -> malloc -> open ->
     libsandbox:open -> dlsym -> malloc -> ...

Change the exec checking logic to scan the ELF instead.  If it exports
these glibc symbols, then we have to assume it can trigger a loop, so
scrub the sandbox environment to prevent us from being loaded.  Then we
use the out-of-process tracer (i.e. ptrace).  This should generally be
as robust anyways ... if it's not, that's a bug we want to fix as this
is the same code used for static apps.

URL: http://crbug.com/586444
Reported-by: Ryo Hashimoto <hashimoto@chromium.org>
Signed-off-by: Mike Frysinger <vapier@gentoo.org>

1 files changed, 44 insertions, 0 deletions

diff --git a/tests/malloc_hooked_tst.c b/tests/malloc_hooked_tst.c
new file mode 100644
index 0000000..18737fe
--- /dev/null
+++ b/tests/malloc_hooked_tst.c
@@ -0,0 +1,44 @@
+/* Make sure programs that override malloc don't mess us up:
+ *
+ * libsandbox's __attribute__((constructor)) libsb_init ->
+ *   libsandbox's malloc() ->
+ *     dlsym("mmap") ->
+ *       glibc's libdl calls malloc ->
+ *         tcmalloc's internal code calls open ->
+ *           libsandbox's open wrapper is hit ->
+ *             libsandbox tries to initialize itself (since it never finished originally) ->
+ *               libsandbox's malloc() ->
+ *                 dlsym() -> deadlock
+ * http://crbug.com/586444
+ */
+
+#include "headers.h"
+
+static void *malloc_hook(size_t size, const void *caller)
+{
+	int urandom_fd = open("/dev/urandom", O_RDONLY);
+	close(urandom_fd);
+	return NULL;
+}
+
+void *(*__malloc_hook)(size_t, const void *) = &malloc_hook;
+
+static void *thread_start(void *arg)
+{
+	return arg;
+}
+
+int main(int argc, char *argv[])
+{
+	/* Make sure we reference some pthread symbols, although we don't
+	 * really want to execute it -- our malloc is limited. */
+	if (argc < 0) {
+		pthread_t tid;
+		pthread_create(&tid, NULL, thread_start, NULL);
+	}
+
+	/* Trigger malloc! */
+	if (malloc(100)) {}
+
+	return 0;
+}