• 3

A PHP Error was encountered

Severity: Notice

Message: Undefined index: userid

Filename: views/question.php

Line Number: 191


File: /home/prodcxja/public_html/questions/application/views/question.php
Line: 191
Function: _error_handler

File: /home/prodcxja/public_html/questions/application/controllers/Questions.php
Line: 433
Function: view

File: /home/prodcxja/public_html/questions/index.php
Line: 315
Function: require_once

I'm trying to run a container on a RHEL 6.5 but I keep hitting this problem:

sudo docker run -u postgres -it registry/postgres /bin/bash
/bin/bash: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: Permission denied

When run as user 'root', the container starts fine but the problem appears again when trying to switch to another user:

$ sudo docker run -u root -it registry/database /bin/bash
[root@8a20410eaa5e /]# su postgres
su: /bin/bash: Permission denied

This is a specific container built by us, based on CentOS 6.5 an that runs Postgres. The Dockerfile to build it has "USER postgres" in it, and it works fine elsewhere except these servers. I can reproduce the same behaviour with a busybox container:

$ sudo docker run -u nobody -it
/ $ ls
/bin/sh: ls: Permission denied

The RHEL 6.5 host has SELinux enabled. We have other other hosts where SELinux and this container works fine there. The audit log for this host looks clean, no error messages that I can see when trying to run the container.

This is what we've tried so far:

  • update the SELinux policies in RHEL ("sudo yum upgrade selinux-policy"), as they were not the latest versions
  • get SELinux into permissive mode (setenforce 0); not tried to switch it off completely and reboot
  • start the Docker daemon with "--selinux-enabled=true"
  • start the container with --privileged
  • start the container with --security-opt=:label:disable
  • we're running the latest RHEL 6.5 kernel: 2.6.32-504.16.2.el6.x86_64

Also run a strace session for the 'su ' command within the container but could not see much beyond these:

 17    setgid(10000)                     = 0
 17    setuid(10000)                     = 0
 17    munmap(0x7f07a3540000, 2101304)   = 0
 17    munmap(0x7f07a311c000, 2113776)   = 0
 17    munmap(0x7f07a2f03000, 2196352)   = 0
 17    munmap(0x7f07a2cea000, 2198192)   = 0
 17    munmap(0x7f07a2ae8000, 2101272)   = 0
 17    munmap(0x7f07a28e4000, 2109624)   = 0
 17    munmap(0x7f07a26e0000, 2109672)   = 0
 17    munmap(0x7f07a24d3000, 2148896)   = 0
 17    munmap(0x7f07a22d0000, 2105488)   = 0
 17    munmap(0x7f07a20cb000, 2113848)   = 0
 17    munmap(0x7f07a1ec5000, 2118168)   = 0
 17    munmap(0x7f07a3321000, 2221912)   = 0
 17    execve("/bin/bash", ["bash"], [/* 15 vars */]) = -1 EACCES (Permission denied)
 17    write(2, "su: ", 4)               = 4
 17    write(2, "/bin/bash", 9)          = 9

The full strace dump is here in case it's needed: http://pastebin.com/42C2B8LP.

We're not sure what to look for next, any ideas?

      • 2
    • have you tried to check you auditd logs? /var/log/audit/audit.log, put selinux in permissive mode, in this way you can see if the problem is selinux.
      • 1
    • We did, both. Setting SElinux to permissive (setenforce 0) and then running the container did not seem to help. I cannot see anything unusual in the audit logs either when running the container: pastebin.com/F6LSUn2J (this is from an unsuccessful run with "sudo docker run -u postgres -it registry/postgres /bin/bash", which returns "/bin/bash: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: Permission denied")
      • 2
    • I also forgot to state that this is happening on Docker 1.6 from EPEL-testing. We did downgrade to 1.3.2 to verify if it was an issue with Docker but the behaviour was still reproducible.
    • @MichaelHampton it's under /var/lib/docker, default location. /var is an LVM volume group: /dev/mapper/vg_main-lv_var on /var type ext4 (rw). It's the same in every host. Or did you mean something else?

I was finally able to solve this problem. Which means that it seems that I found a solution, but I'm not still sure what the problem is:

1) pull container from registry 2.0 + run with docker 1.6 -> fail

2) pull container from registry 0.9.x (either Docker's own or the Artifactory server that we run internally) + run with docker 1.6 -> works

3) pull container from registry 2.0 + run with docker 1.5 or older -> fail

4) pull container form registry 0.9.x + run with docker 1.5 or newer -> works

I don't really think that Registry 2.0 is at fault but I have no better answer. The new registry is much faster than the old one but guess we'll stick with the older registry for the time being.

  • 0
Reply Report
    • Did you find out the root cause? The same thing happened to me RHEL 6.8, docker 1.7.1. How did you force docker to pull from v1?

Trending Tags