Testing Software Firewall Versus Router Firewall

I was curious to see how secure my firewall was on my linux box and I changed some settings on my router to point the DMZ to my linux box and then did some port probes with Shields UP port probe. Obviously I had port 80 open (because I have a http server running - but usually use it only for testing website designs and block it at the router most of the time). But I was curious as to whether or not this was an effective way to test my linux box's firewall.
Also, I use a couple ports for peer-to-peer file sharing, but rarely use these ports because I rarely run the peer-to-peer software.
Obviously the one tcp port was open too.
I was wondering (aside from this being a good way to test the software firewall) is there a way to automatically open the one tcp port automatically only when I run the peer-to-peer software. I'd really rather they be stealthed unless I have a need for them and don't really want to have to open my firewall settings and manually turn them on and off everytime I want to use peer-to-peer.
Or am I just being too paranoid?
On the same note, is it pointless to scan every port possible just to see or should I feel secure just knowing what ports I have manually set as open in my software firewall?
I am paranoid right?

I was curious to see how secure my firewall was on my linux box and I changed some settings on my router to point the DMZ to my linux box and then did some port probes with Shields UP port probe. Obviously I had port 80 open (because I have a http server running - but usually use it only for testing website designs and block it at the router most of the time). But I was curious as to whether or not this was an effective way to test my linux box's firewall.
Also, I use a couple ports for peer-to-peer file sharing, but rarely use these ports because I rarely run the peer-to-peer software.
Obviously the one tcp port was open too.
I was wondering (aside from this being a good way to test the software firewall) is there a way to automatically open the one tcp port automatically only when I run the peer-to-peer software. I'd really rather they be stealthed unless I have a need for them and don't really want to have to open my firewall settings and manually turn them on and off everytime I want to use peer-to-peer.
Or am I just being too paranoid?
On the same note, is it pointless to scan every port possible just to see or should I feel secure just knowing what ports I have manually set as open in my software firewall?
I am paranoid right?
[/b]

Hi,

Answer to you question is "NO", however there is something that works other way around called "port knocking". I havent heard of reverse port knocking daemons, but they might exist ?, and besides your ptp client sopen those ports only when they need to connect to the network of your peers (it's not that they are open at all times - if they are, throw that software to trash).