Strange behavior on openvz VPS

vilanye · June 15, 2016, 6:32pm

I have had a 13.1 server running for a few years with no problems. Yesterday my web app was unable to access any external webservice, I couldn’t even ping an ip address. I didn’t change anything.

I turned off the firewall to allow the services to be accessed. For some reason the network was not assigned a zone, I reassigned it external where I allow 22, 80, 443 and 9292 incoming. But when I turn the firewall back on everything returns filtered on a nmap scan and all outgoing traffic is blocked.

I am not sure where to go from here. Here is the ifconfig output:


lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:21647 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21647 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:7235837 (6.9 Mb)  TX bytes:7235837 (6.9 Mb)

venet0    Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:127.0.0.1  P-t-P:127.0.0.1  Bcast:0.0.0.0  Mask:255.255.255.255
          UP BROADCAST POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
          RX packets:119174 errors:0 dropped:0 overruns:0 frame:0
          TX packets:98040 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:109418356 (104.3 Mb)  TX bytes:43679842 (41.6 Mb)

venet0:0  Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:xxx.xxx.232.225  P-t-P:xxx.xxx.232.225  Bcast:0.0.0.0  Mask:255.255.255.255
          UP BROADCAST POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1

iptables when the firewall is turned on.


iptables -L
Chain INPUT (policy DROP)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere            
input_dmz  all  --  anywhere             anywhere            
input_ext  all  --  anywhere             anywhere            
LOG        all  --  anywhere             anywhere             limit: avg 3/min burst 5 LOG level warning tcp-options ip-options prefix "SFW2-IN-ILL-TARGET "
DROP       all  --  anywhere             anywhere            

Chain FORWARD (policy DROP)
target     prot opt source               destination         
LOG        all  --  anywhere             anywhere             limit: avg 3/min burst 5 LOG level warning tcp-options ip-options prefix "SFW2-FWD-ILL-ROUTING "

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere            

Chain forward_dmz (0 references)
target     prot opt source               destination         

Chain forward_ext (0 references)
target     prot opt source               destination         

Chain input_dmz (1 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere             PKTTYPE = broadcast
ACCEPT     icmp --  anywhere             anywhere             icmp source-quench
ACCEPT     icmp --  anywhere             anywhere             icmp echo-request
DROP       all  --  anywhere             anywhere             PKTTYPE = multicast
DROP       all  --  anywhere             anywhere             PKTTYPE = broadcast
LOG        tcp  --  anywhere             anywhere             limit: avg 3/min burst 5 tcp flags:FIN,SYN,RST,ACK/SYN LOG level warning tcp-options ip-options prefix "SFW2-INdmz-DROP-DEFLT "
LOG        icmp --  anywhere             anywhere             limit: avg 3/min burst 5 LOG level warning tcp-options ip-options prefix "SFW2-INdmz-DROP-DEFLT "
DROP       all  --  anywhere             anywhere            

Chain input_ext (1 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere             PKTTYPE = broadcast
ACCEPT     icmp --  anywhere             anywhere             icmp source-quench
ACCEPT     icmp --  anywhere             anywhere             icmp echo-request
LOG        tcp  --  anywhere             anywhere             limit: avg 3/min burst 5 tcp dpt:ssh flags:FIN,SYN,RST,ACK/SYN LOG level warning tcp-options ip-options prefix "SFW2-INext-ACC-TCP "
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:http
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:armtechdaemon
DROP       all  --  anywhere             anywhere             PKTTYPE = multicast
DROP       all  --  anywhere             anywhere             PKTTYPE = broadcast
LOG        tcp  --  anywhere             anywhere             limit: avg 3/min burst 5 tcp flags:FIN,SYN,RST,ACK/SYN LOG level warning tcp-options ip-options prefix "SFW2-INext-DROP-DEFLT "
LOG        icmp --  anywhere             anywhere             limit: avg 3/min burst 5 LOG level warning tcp-options ip-options prefix "SFW2-INext-DROP-DEFLT "
DROP       all  --  anywhere             anywhere            

Chain reject_func (0 references)
target     prot opt source               destination         
REJECT     tcp  --  anywhere             anywhere             reject-with tcp-reset
REJECT     udp  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-proto-unreachable

tsu2 · June 15, 2016, 7:08pm

Verify your venet0:0 is a bridge device, it should be listed when you run the following command

brctl show

I’ve sometimes seen a bridge device configured, but the default gateway not set to use that bridge device interface.
Open
YAST > Network Devices > Routing tab

The IPv4 interface “Device” value might be empty or mis-configured.
If you click on on the IPv4 “Device” you should see a dropdown and venet0:0 should be listed.
Select that and save.

If YAST doesn’t automatically restart your network services, do so.
Don’t remember for sure but I think that 13.1 did implement systemd to manage networking, so

systemctl restart network

Your system should be functioning fine.

TSU

vilanye · June 15, 2016, 7:47pm

Thanks for your response. brctl show returns nothing.

In yast,there is nothing in network settings. The dropdown box in the routing tab only shows lo and venet0.

tsu2 · June 15, 2016, 8:17pm

OK,
Then it’s probably time to investigate the history of this machine, where was the original image created from, something provided to you or something you created?

You provided ifconfig results, but what might be in

ip addr

and

ip route

If your machine’s image originally came from your Provider,
How complex is your machine, would it be difficult to migrate your apps and data to a newly provided image?
If you can even launch a newly provided image without your apps, you should have a working network configuration to compare.
How difficult would it be to roll back to before you noticed problems, often times virtual machines can be backed up quickly and easily by simply cloning the machine periodically.

Also, I’d be curious what interface configuration interfaces are currently listed,

ls /etc/sysconfig/network/

Also, is this a private openVZ deployment or one on a commercial Provider?

TSU

vilanye · June 15, 2016, 8:28pm

The image came from the host. It was actually a 12.3 image that I upgraded. The host is http://nosupportvpshosting.com/.


ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: venet0: <BROADCAST,POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/void
    inet 127.0.0.1/32 scope host venet0
    inet xxx.xxx.232.225/32 scope global venet0:0


ip route
127.0.0.0/8 dev lo  scope link
default dev venet0  scope link


ls /etc/sysconfig/network/
config  if-down.d  ifcfg-lo      ifcfg.template  ifroute-venet0  scripts
dhcp    if-up.d    ifcfg-venet0  ifroute-lo      providers

I am thinking of finding a host with more up to date images and actual support since I am not a professional admin. Thank you for your time.

tsu2 · June 15, 2016, 8:36pm

Some stuff to chew on…

I haven’t personally worked with openVZ in awhile, so at least for personal reasons I looked up some current openVZ network documentation…

Depending on where you’re pulling your updates from (openVZ as well as OSS?), could your networking have changed technologies?

The general openVZ networking articles
https://openvz.org/Category:Networking

veth(deprecated) vs venet(current)
https://openvz.org/Differences_between_venet_and_veth

Managing and configuring venet from the command line
https://openvz.org/Virtual_network_device

So, it appears that venet does not support bridge devices of any type and consistent with your brctl result…
Do you have control over the Host or only inside the container?
I’m suspecting at this point possibly at this point that the proper solution might be to remove and re-create network interfaces in your openVZ container altogether, but before doing something like this, one would have to closely read and understand the current openVZ networking architecture, for instance if it has any similarities with Docker containers (which could have provided an inspiration in either direction) which requires configuring the “outside” interface on the Host to match the “inside” interface in the container. At least in Docker, the name of an interface does not seem to matter, it only matters understanding and correctly detailing what is exposed (typically all the network properties including networkID and port).

If openVZ networking is indeed similar to Docker networking, then you might also consider that really basic iptables filtering is relatively useless, the pin-holes you create to enable network connections do essentially the same thing.

TSU