Running apache when using an openvpn connection

English Network/Internet
1

I’m using an openvpn account for internet access but want to be able to connect to ssh and apache from external locations

locally over the lan everything’s working ok, ssh with passwordless logins using a custom port and apache with name-based hosting all function ok using the machine’s lan ip

If I try connecting from an external location though, no such luck. I’ve tried using my external ip as assigned by my isp and also the ip assigned by my openvpn account provider and both time out

If I try the isp-assigned ip from another machine on the lan connection works, whereas not with the openvpn-assigned ip which I pretty much expected

I have a dyndns account resolving a domain to my isp ip, again this works locally but not externally

With the openvpn connection turned off, accessing the machine with the ip address and domain names from an external location both work fine which I take to mean it’s pretty much certain that the vpn connection is causing the issue

Anyone know if there’s some configuration I need to do to make this work or am I always gonna have this issue when using the vpn connection no matter what I do?

2

Have you tried watching the wire to see if the packets from the outside
sources are at least getting to your box when VPN’d? I’ve seen cases
where extra routes (like those from a VPN) will cause packets to be
returned via the wrong path which can cause issues. If you see the packet
come in, and then go back out, that could help determine what to do next:

sudo /usr/sbin/tcpdump -n -s 0 -i any port 80

While running that do your tests from outside your LAN to see if the
packets make it in to your machine at all. The Apache/SSH logs probably
wouldn’t be enough at this point since they typically show what happens
after TCP has established a connection, and it doesn’t sound like you’re
getting that far in these tests.

Good luck.

3

Thanks for the reply there ab, didn’t see it until now as I had to go out

Here’s the tcpdump output from attempting to connect using both the android stock browser and opera mobile, when I got ‘connection refused domain.com:80’

ecky@linux:~> sudo /usr/sbin/tcpdump -n -s 0 -i any port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
20:28:20.521636 IP 31.120.190.181.45249 > 192.168.0.20.80: Flags [S], seq 3128318783, win 49640, options [mss 1460,nop,wscale 0,nop,nop,sackOK], length 0
20:28:23.894029 IP 31.120.190.181.45249 > 192.168.0.20.80: Flags [S], seq 3128318783, win 49640, options [mss 1460,nop,wscale 0,nop,nop,sackOK], length 0
20:28:30.342229 IP 31.120.190.181.60523 > 192.168.0.20.80: Flags [S], seq 1576898385, win 49640, options [mss 1460,nop,wscale 0,nop,nop,sackOK], length 0
20:28:33.700004 IP 31.120.190.181.60523 > 192.168.0.20.80: Flags [S], seq 1576898385, win 49640, options [mss 1460,nop,wscale 0,nop,nop,sackOK], length 0
20:28:39.616834 IP 31.120.190.181.60523 > 192.168.0.20.80: Flags [R], seq 1576898386, win 49640, length 0
20:30:34.956344 IP 80.239.242.158.59232 > 192.168.0.20.80: Flags [S], seq 4029635346, win 5840, options [mss 1460,sackOK,TS val 2567790399 ecr 0,nop,wscale 7], length 0
20:30:37.965150 IP 80.239.242.158.59232 > 192.168.0.20.80: Flags [S], seq 4029635346, win 5840, options [mss 1460,sackOK,TS val 2567791149 ecr 0,nop,wscale 7], length 0
20:30:43.955299 IP 80.239.242.158.59232 > 192.168.0.20.80: Flags [S], seq 4029635346, win 5840, options [mss 1460,sackOK,TS val 2567792649 ecr 0,nop,wscale 7], length 0
20:30:58.161439 IP 80.239.242.158.52502 > 192.168.0.20.80: Flags [S], seq 532902308, win 5840, options [mss 1460,sackOK,TS val 2567796201 ecr 0,nop,wscale 7], length 0
20:31:01.161523 IP 80.239.242.158.52502 > 192.168.0.20.80: Flags [S], seq 532902308, win 5840, options [mss 1460,sackOK,TS val 2567796951 ecr 0,nop,wscale 7], length 0
20:31:07.161438 IP 80.239.242.158.52502 > 192.168.0.20.80: Flags [S], seq 532902308, win 5840, options [mss 1460,sackOK,TS val 2567798451 ecr 0,nop,wscale 7], length 0

Nothing shows in apache’s access and error logs

4

So apparently your IP address is 192.168.0.20, or at least one of them is.
Is this the VPN address? Have you tried restarting your desired services
(httpd/sshd) after starting the VPN? Are those services set to listen on
all address or only one one or two?

That you are getting a connection refused (vs. nothing) makes me think
that if the services are running they are just not binding to the new IP
address to listen, which might make sense depending on their
configuration. What is the output of the following before and after
setting up the VPN connection (assuming all the while both httpd and sshd
are running):

netstat -planet | grep 'LISTEN ’

The output you’ve provided so far at least confirms that packets are
getting to the box, but nothing is going back out, presumably because
nothing is accepting the packets coming in (not listening for them).

Good luck.

5

Here’s the output from ifconfig:

eth1      Link encap:Ethernet  HWaddr D8:5D:4C:81:49:F6  
          inet addr:192.168.0.20  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::da5d:4cff:fe81:49f6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8260852 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10958236 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:5061403043 (4826.9 Mb)  TX bytes:11970887956 (11416.3 Mb)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1169986 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1169986 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:10332490597 (9853.8 Mb)  TX bytes:10332490597 (9853.8 Mb)

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:10.167.3.6  P-t-P:10.167.3.5  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:1302284 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1265624 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:917868447 (875.3 Mb)  TX bytes:530367406 (505.7 Mb)

Netstat without openvpn [irrelevant info removed]:


tcp        0      0 0.0.0.0:5000            0.0.0.0:*               LISTEN      0          17343      4572/sshd  
tcp        0      0 :::5000                 :::*                    LISTEN      0          17353      4572/sshd 
tcp        0      0 :::80                   :::*                    LISTEN      0          35699      7410/httpd2-prefork

With openvpn running the netstat output remains the same

6

I’ve just noticed I’m getting a lot of these in /var/log/messages:

Jan  2 00:27:51 linux kernel: [121818.938740] IPv4: martian source 192.168.0.20 from 116.232.135.72, on dev eth1

I believe these messages indicate an issue with the network setup?

I don’t have a good understanding of how vpn tunneling works as this is the first time I’ve ever tried using vpn, but I took a look at the routing table without the vpn connection running and there is something there that I’m not sure about


/sbin/route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth1
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth1
184.95.54.226   192.168.0.1     255.255.255.255 UGH   0      0        0 eth1
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
198.24.140.58   192.168.0.1     255.255.255.255 UGH   0      0        0 eth1

184.95.54.226 is my vpn provider’s server so I suppose it makes sense to have an entry for that, but should it be there when openvpn is stopped?

198.24.140.58 would have been an ip assigned when the vpn connection is running so it seems to me that shouldn’t be there?

Here’s the table when openvpn is running

/sbin/route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.194.146.5    128.0.0.0       UG    0      0        0 tun0
0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth1
10.194.146.1    10.194.146.5    255.255.255.255 UGH   0      0        0 tun0
10.194.146.5    0.0.0.0         255.255.255.255 UH    0      0        0 tun0
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
128.0.0.0       10.194.146.5    128.0.0.0       UG    0      0        0 tun0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth1
184.95.54.226   192.168.0.1     255.255.255.255 UGH   0      0        0 eth1
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
198.24.140.58   192.168.0.1     255.255.255.255 UGH   0      0        0 eth1
7

Well, let’s start with the things I know. Your SSH service is explicitly
set to listen on TCP 5000 for both IPv4 and IPv6 protocols, so that’s
nice. Your httpd service, though, only shows within netstat as listening
on TCP 80 for IPv6. That’s not uncommon, and has to do with magic in the
kernel and how the handle to the socket is grabbed in the application’s
call to the OS for the same, but it’s interesting. Since neither of these
work when you fire up OpenVPN that difference is probably not relevant.

Now, just as a sidenote, ‘ifconfig’ is really, really, really old and was
deprecated years ago; the ‘ip’ command has replaced it, and ‘route’, and
‘arp’, and all kinds of things so it (‘ip’) should be used instead in all
cases. This is nice for a couple of reasons; first, it behaves better in
many cases; second, it works in many cases where other things (including
ifconfig) do not; third, it’s shorter; fourth, it’s in the non-root user’s
PATH by default so it doesn’t require relative/absolute paths or running
as ‘root’. Anyway, here is what I’d like to see:

ip addr
ip route
ip -s link
ip neigh

Feel free to compare the output of these commands with the old, obsoleted
commands like ifconfig just to see how things are represented differently
in some cases.

One thing that does seem a little interesting is that your VPN address
does not have an IPv6 address like your standard NIC (eth1). Will that
interfere with packets getting to the standard NIC’s IP address after
OpenVPN starts? I don’t know why it would, but the OS is complex enough
that I definitely do not know everything. Some other output that may be
interesting:

#get firewall rules, in case the firewall is up at all.
/usr/sbin/iptables-save

It may also be interesting to hear about restarting Apache httpd after
OpenVPN is running to see if that changes anything. In the conf file is
the Listen line defined to listen on all IPs, and what if you set it
explicitly to your IPv4 address bound to eth1?

Good luck.

8

One of my posts seems to have disappeared, in the netstat I now have this entry

tcp        0      0 192.168.0.20:80         0.0.0.0:*               LISTEN      0          9684       1253/httpd2-prefork

In apache’s listen.conf yast had added (I assume) this line

Listen 80

I changed it to:

Listen 192.168.0.20:80

With regards the routing table, I think the entries I mentioned in my last post were ‘left over’ as I had issued the command after openvpn had started and then been stopped. I rebooted with openvpn disabled then started it manually and the table then looks like this:

/sbin/route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth1
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth1
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1

ecky@linux:~> sudo /etc/init.d/openvpn start
redirecting to systemctl

ecky@linux:~> /sbin/route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.186.142.9    128.0.0.0       UG    0      0        0 tun0
0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth1
10.186.142.1    10.186.142.9    255.255.255.255 UGH   0      0        0 tun0
10.186.142.9    0.0.0.0         255.255.255.255 UH    0      0        0 tun0
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
128.0.0.0       10.186.142.9    128.0.0.0       UG    0      0        0 tun0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth1
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
198.15.98.58    192.168.0.1     255.255.255.255 UGH   0      0        0 eth1
9

Admitting defeat on this

Tried everything I could think of, read more search results on running apache through a vpn connection than I can count and tried everything I found in them that made sense in my situation that I hadn’t already thought of (which wasn’t much), tried different ports even though my provider assured me they weren’t blocking any, nothing worked

Must be at least one way of doing this but I’m darned if I can find one

10

If you want to look at this a little longer, I’m willing to think about what you’re doing…

For starters, IMO you don’t describe your setup nearly well enough, particularly how you have your VPN setup and between hosts in what locations.

Is your Server multi-homed, or is your Server single homed and your Gateway is translating your public IP to your Server’s private IP?
Is your Server located in the same network as your LAN or on the Edge either directly exposed to the Internet or in a DMZ?

Are any Proxy or Reverse Proxy devices involved?

When you say you use openVPN for Internet Access, is that only for clients? Somehow I get the picture you’re not using a VPN when exposing your Apache server (yes, it’s a not uncommon config particularly in countries where server access is restricted. See Tor documentation and uses).

If you’re using OpenVPN only for client access outbound to the Internet, then what is your remote endpoint? –
Or are you actually trying to connect directly to your own Server by way of the external interface?

This is all important to understand whether as I suspect you may be creating a NetworkID conflict which would kill any routing.

And, those are just the starting questions to understand your basic setup without <then> looking at the network settings for relevant devices and connections.

TSU

11

Always willing to give it a go

The server isn’t ‘my’ VPN, it’s basically an account I rent for providing anonymity etc so I have no control over any settings on the server, as for the client I’m just starting openvpn as a service and have the following client.conf file in /etc/openvpn:


dev tun
proto udp
remote us-west.privateinternetaccess.com 1194
resolv-retry infinite
nobind
user nobody
group nobody
persist-key
persist-tun
ca ca.crt
auth-user-pass login.info
ns-cert-type server
comp-lzo
verb 3
redirect-gateway

Not sure what you mean by that [machine running apache or the vpn server, and what’s meant by multi-homed’

If by ‘Server’s private IP’ you mean my apache’s ip on the lan then yes

Again, not sure whether you mean the vpn server or apache, vpn isn’t on the lan, apache is, no dmz for apache & don’t know about VPn, don’t know what you mean by ‘on the Edge’

Not at my end, can’t speak for the vpn provider

Yes, just clients

I want to be able to run apache whilst using the vpn connection, couldn’t see anything of use in the tor docs unless I just didn’t recognise it when I was reading

Not entirely sure what you mean by remote endpoint, but if you mean the vpn server’s host it’s: us-west.privateinternetaccess.com [ip: 184.95.54.226]

I’m gonna have to take your word for that mate as the whole issue seems a bit beyond my understanding at this point

Any other questions/info you need just ask, really appreciate you taking the time to try helping me with this TSU

I could run the apache, which only has very light traffic anyway, on another machine without using the vpn connection on that one but that would be somewhat inconvenient

12

A multi-homed server has multiple NICS, this is important because oftentimes the machine has one NIC connecting to the Internet and the other to the LAN. I’m going to assume then that your Apache server only has one NIC.

Pls verify then that the IP address on your Apache box is configured with a LAN address, and your Internet Gateway is translating your Public IP address to the Private IP address on your Apache box. The alternative is for your Apache box to be configured with the Public IP address bound to the website exposed to the Internet.

I do understand now how you use and what your outbound VPN is.

Believe we may be surrounding the issue. Will have to review in my mind the implications that your Gateway is not a Reverse Proxy and is likely translating your Apache website configured with an IP address using the same NetworkID as your client machine. If you were able to configure your Apache website the second way I suggested with the Public IP address and your Gateway simply passed traffic to your website, there would not be any question things should work.

If you’re concerned simply with accessing the website and not making exteror access using the Public IP a requirement, the simplest solution is to configure what is known as a “split DNS” – In your LAN DNS you simply make a copy of your Domain’s Public Records but modify them with appropriate Private Addresses. The result is that clients from outside your LAN will continue to use the Public IP address but clients in your LAN would be pointed to Private IP addresses for the same resources instead.

TSU