Results 1 to 5 of 5

Thread: Docker container external internet connectivity issues

  1. #1

    Question Docker container external internet connectivity issues

    Hello all,

    First post, but hopefully will get involved in other bits after this... Caveat - I'm not a networking boffin.l have experience with linux + docker, but not super hot.
    ---------

    OS: Tumbleweed 20170505

    Docker version:
    Code:
    Client:
     Version:      1.13.0
     API version:  1.25
     Go version:   go1.7
     Git commit:   78d1802
     Built:        Mon Apr 17 09:04:48 2017
     OS/Arch:      linux/amd64
    
    Server:
     Version:      1.13.0
     API version:  1.25 (minimum version 1.12)
     Go version:   go1.7
     Git commit:   78d1802
     Built:        Mon Apr 17 09:04:48 2017
     OS/Arch:      linux/amd64
     Experimental: false
    Can pull and run docker images fine - e.g. hello-world, redis, ubuntu, alpine. However, containers can't connect to the internet to, for example, perform apt-get or pip install. Happens in both docker builds or docker runs. Ran the following and no return packets:

    Code:
    dijksterhuis@pc-67-96:~> docker run --rm -it alpine '/bin/ash'                                
    / # ping google.com
    PING google.com (216.58.212.110): 56 data bytes

    Also ran following to check above wasn't dodgy alpine container.
    Code:
    
    dijksterhuis@pc-67-96:~> docker run --name test -it ubuntu "/bin/bash"  
    root@ea7f1bb9646c:/# apt-get update
    Get:1 http://archive.ubuntu.com/ubuntu xenial InRelease [2127 B]
    Get:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [2127 B]
    Get:4 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [2127 B]
    Err:1 http://archive.ubuntu.com/ubuntu xenial InRelease
      Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?)
    Err:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease
      Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?)
    Err:4 http://archive.ubuntu.com/ubuntu xenial-backports InRelease
      Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?)
    Get:2 http://security.ubuntu.com/ubuntu xenial-security InRelease [2127 B]
    Err:2 http://security.ubuntu.com/ubuntu xenial-security InRelease
      Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?)
    Fetched 4254 B in 0s (71.9 kB/s)    
    Reading package lists... Done
    E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial/InRelease  Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?)
    E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial-updates/InRelease  Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?)
    E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial-backports/InRelease  Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?)
    E: Failed to fetch http://security.ubuntu.com/ubuntu/dists/xenial-security/InRelease  Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?)
    E: Some index files failed to download. They have been ignored, or old ones used instead.
    My ISP set up 1:2:1 NAT for me last night to allow me to SSH into my desktop. Thought it could be an issue so have asked them to disable and rebooted. No luck. I do know for a fact that they don't like port forwarding (incidentally, they won't allow SSH on port 22, SSH must be on a port higher than 1024 - could this be something to do with it?).

    Attempted same tests on Macbook Pro (same network) and worked fine. So don't believe it's external network issue. Tried zypper rm docker ; zypper in docker. No luck. systemctl restart docker has no effect on issue.

    Scoured some github issues and looked at tcpdump and iptables. For tcpdump, ran:
    Code:
    
    sudo tcpdump -i docker0 icmp -v
    sudo  tcpdump -i enp2s0 icmp -v 
    sudo tcpdump -i veth537b204 icmp -v

    Only successful packet transfers I saw were to google. Nothing seems to returns successfully (port unreachable error?)


    tcpdump docker0
    Code:
    18:52:23.188925 IP (tos 0x0, ttl 64, id 45280, offset 0, flags [DF], proto ICMP (1), length 84)
        172.17.0.2 > google-public-dns-a.google.com: ICMP echo request, id 2816, seq 53, length 64
    18:52:23.189163 IP (tos 0xc0, ttl 63, id 40164, offset 0, flags [none], proto ICMP (1), length 112)
        gw.customer.marketgait.collegiate.lan > 172.17.0.2: ICMP google-public-dns-a.google.com protocol 1 port 27949 unreachable, length 92
            IP (tos 0x0, ttl 62, id 45280, offset 0, flags [DF], proto ICMP (1), length 84)
        172.17.0.2 > google-public-dns-a.google.com: ICMP echo request, id 2816, seq 53, length 64
    18:52:24.189053 IP (tos 0x0, ttl 64, id 45346, offset 0, flags [DF], proto ICMP (1), length 84)
        172.17.0.2 > google-public-dns-a.google.com: ICMP echo request, id 2816, seq 54, length 64
    18:52:24.189372 IP (tos 0xc0, ttl 63, id 40168, offset 0, flags [none], proto ICMP (1), length 112)
        gw.customer.marketgait.collegiate.lan > 172.17.0.2: ICMP google-public-dns-a.google.com protocol 1 port 40169 unreachable, length 92
            IP (tos 0x0, ttl 62, id 45346, offset 0, flags [DF], proto ICMP (1), length 84)
        172.17.0.2 > google-public-dns-a.google.com: ICMP echo request, id 2816, seq 54, length 64
    tcmpdump enp2s0
    Code:
    18:56:14.214688 IP (tos 0x0, ttl 63, id 9117, offset 0, flags [DF], proto ICMP (1), length 84)
        pc-67-96.customer.marketgait.collegiate.lan > google-public-dns-a.google.com: ICMP echo request, id 2816, seq 284, length 64
    18:56:14.214929 IP (tos 0xc0, ttl 64, id 40640, offset 0, flags [none], proto ICMP (1), length 112)
        gw.customer.marketgait.collegiate.lan > pc-67-96.customer.marketgait.collegiate.lan: ICMP google-public-dns-a.google.com protocol 1 port 23308 unreachable, length 92
            IP (tos 0x0, ttl 62, id 9117, offset 0, flags [DF], proto ICMP (1), length 84)
        pc-67-96.customer.marketgait.collegiate.lan > google-public-dns-a.google.com: ICMP echo request, id 2816, seq 284, length 64
    18:56:15.214815 IP (tos 0x0, ttl 63, id 9190, offset 0, flags [DF], proto ICMP (1), length 84)
        pc-67-96.customer.marketgait.collegiate.lan > google-public-dns-a.google.com: ICMP echo request, id 2816, seq 285, length 64
    18:56:15.215117 IP (tos 0xc0, ttl 64, id 40641, offset 0, flags [none], proto ICMP (1), length 112)
        gw.customer.marketgait.collegiate.lan > pc-67-96.customer.marketgait.collegiate.lan: ICMP google-public-dns-a.google.com protocol 1 port 36296 unreachable, length 92
            IP (tos 0x0, ttl 62, id 9190, offset 0, flags [DF], proto ICMP (1), length 84)
        pc-67-96.customer.marketgait.collegiate.lan > google-public-dns-a.google.com: ICMP echo request, id 2816, seq 285, length 64
    e.g. veth537b204
    Code:
    18:57:07.221139 IP (tos 0x0, ttl 64, id 16578, offset 0, flags [DF], proto ICMP (1), length 84)
        172.17.0.2 > google-public-dns-a.google.com: ICMP echo request, id 2816, seq 337, length 64
    18:57:07.221367 IP (tos 0xc0, ttl 63, id 40695, offset 0, flags [none], proto ICMP (1), length 112)
        gw.customer.marketgait.collegiate.lan > 172.17.0.2: ICMP google-public-dns-a.google.com protocol 1 port 41987 unreachable, length 92
            IP (tos 0x0, ttl 62, id 16578, offset 0, flags [DF], proto ICMP (1), length 84)
        172.17.0.2 > google-public-dns-a.google.com: ICMP echo request, id 2816, seq 337, length 64
    18:57:08.221252 IP (tos 0x0, ttl 64, id 16721, offset 0, flags [DF], proto ICMP (1), length 84)
        172.17.0.2 > google-public-dns-a.google.com: ICMP echo request, id 2816, seq 338, length 64
    iptables output (not sure what looking at here). Entries that see packet increases marked (!)
    Code:
    
    dijksterhuis@pc-67-96:~> clear ; sudo iptables -t filter -nvL
    Chain INPUT (policy DROP 0 packets, 0 bytes)
     pkts bytes target     prot opt in     out     source               destination          
        0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0            
    (!) 62242   70M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate ESTABLISHED
        0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED
       17  3258 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:5353 PKTTYPE = multicast
       11  4754 input_ext  all  --  *      *       0.0.0.0/0            0.0.0.0/0            
        0     0 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 5 LOG flags 6 level 4 prefix "SFW2-IN-ILL-TARGET "
        0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            
    
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
     pkts bytes target     prot opt in     out     source               destination          
    (!)  284 34620 DOCKER-ISOLATION  all  --  *      *       0.0.0.0/0            0.0.0.0/0            
    (!)  141 22735 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            
    (!)  141 22735 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    (!)  143 11885 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0            
        0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0            
        0     0 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 5 LOG flags 6 level 4 prefix "SFW2-FWD-ILL-ROUTING "
    
    Chain OUTPUT (policy ACCEPT 31759 packets, 3621K bytes)
     pkts bytes target     prot opt in     out     source               destination          
        0     0 ACCEPT     all  --  *      lo      0.0.0.0/0            0.0.0.0/0            
    
    Chain DOCKER (1 references)
     pkts bytes target     prot opt in     out     source               destination          
    
    Chain DOCKER-ISOLATION (1 references)
     pkts bytes target     prot opt in     out     source               destination          
      284 34620 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            
    
    Chain forward_ext (0 references)
     pkts bytes target     prot opt in     out     source               destination          
    
    Chain input_ext (1 references)
     pkts bytes target     prot opt in     out     source               destination          
        6  4110 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            PKTTYPE = broadcast
        0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 4
        0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 8
        0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* sfw2.insert.pos */ PKTTYPE != unicast
        0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 5 tcp flags:0x17/0x02 LOG flags 6 level 4 prefix "SFW2-INext-DROP-DEFLT "
        0     0 LOG        icmp --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 5 LOG flags 6 level 4 prefix "SFW2-INext-DROP-DEFLT "
        0     0 LOG        udp  --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 5 ctstate NEW LOG flags 6 level 4 prefix "SFW2-INext-DROP-DEFLT "
        5   644 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            
    
    Chain reject_func (0 references)
     pkts bytes target     prot opt in     out     source               destination          
        0     0 REJECT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with tcp-reset
        0     0 REJECT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
        0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-proto-unreachable


    Any suggestions or help would be much appreciated! At my wits end. I'd rather not switch distros, Tumbleweed is really growing on me.

    Thanks in advance for any help,

    Dijksterhuis

  2. #2

    Default SOLVED: Docker container external internet connectivity issues

    Solved (sort of) - discovered the issue here when I was posting the above on stackoverflow. Will be running with -net=host arg for now... not particularly sure about sockets. If someone has any ideas about that, would love some help/pointers...

    http://stackoverflow.com/questions/3...cess-internet#

  3. #3
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    11,306
    Blog Entries
    2

    Default Re: Docker container external internet connectivity issues

    Take a look at my Docker "How To" that improved on content that was posted at docker.com. Although things have changed slightly, most of the content on my page is still relevant, like one way to configure networking. Note that there are <many> networking concepts if you want to master (or be versatile) configuring networking (The problem with your command is that you didn't define a networking interface as you discovered). Don't know what you mean by "sockets" -- network sockets? something else?

    https://github.com/putztzu/docker/bl.../linux/SUSE.md

    Note the concepts... What kind of networking interface to configure, and if you want to configure inbound connections you have to configure forwarding from the Host to the container (you don't have to configure this if you only need to enable outbound connections).

    If you want to run through some additional quick exercises, I adapted a number of older docker demos to openSUSE commands which can be found listed on this page (along with a bunch of other stuff I've written about), all useful stuff if you're brand new to docker...

    https://en.opensuse.org/User:Tsu2

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  4. #4

    Default Re: Docker container external internet connectivity issues

    @tsu2

    Yep, saw your page but only glossed over it in early research - didn't think it would apply or something. Silly me! Will check out the networking stuff you talk about when I've got some spare time. the --net=host arg hack should work for now, but not ideal.

    Plenty of experience with docker, just on my Macbook. Thanks for the links though. Bookmarked them.

  5. #5
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    11,306
    Blog Entries
    2

    Default Re: Docker container external internet connectivity issues

    Quote Originally Posted by dijksterhuis View Post
    @tsu2

    Yep, saw your page but only glossed over it in early research - didn't think it would apply or something. Silly me! Will check out the networking stuff you talk about when I've got some spare time. the --net=host arg hack should work for now, but not ideal.

    Plenty of experience with docker, just on my Macbook. Thanks for the links though. Bookmarked them.
    Yes,
    Actually setting "-net=host" is an early configuration not suggested often in current documentation, I offered that as a "quick configuration" because it's fast, easy to configure and "just works."

    Links to "more normal" networking when the container has its own networking interface are in the links I provided in that same section.

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •