I have mentioned in the past that I get my internet from my cell phone…this results in faster than dialup speeds but slower than a good dsl connection…roughly 25k per second downloads on an edge connection with T-Mobile. 3G is not yet available in my area for a faster connection. Over the years I have played with various cacheing proxies in the hopes of speeding up my surfing…wwwoffle, privoxy (for ad filtering), polipo…etc…recently I decided to create one master cache for the entire house and setup transparent proxying…getting the traffic to my pc was easy by setting my wifi router to forward all port 80 traffic to my main pc. I’ve always done adfiltering to not download any more than I have to and I wanted to do those same things. I did a little bit of playing around with both squid 2.7 and squid 3.0…the configuration files are different between the two versions so you can’t share the config files. I decided to use “adzap” as the squid plugin to do ad blocking…and as time went by I noticed if I was downloading rpm’s from “download.opensuse.org” and it directed me to mirror A and my pc in front office got sent to mirror B it tried to download the files all over again…this resulted in alot of wasted space in the cache. After some googling I came across a product used by fedora users called “intelligentmirrors”…it worked as a plugin for squid and basically any time it encountered an rpm it would store it under a new url pointing to the local machine…this allowed you to build a cache and if you had to update other pc’s at same house they’d get fed all the files from the cache. It was exactly what I wanted but the latest most efficient version only runs on squid 2.7…not the 3.0 I had upgraded to…so once again I downgraded…got transparent proxying working…added in adzap so the ads would get blocked and played with intelligentmirror. It promptly killed squid. 
intelligentmirror was written by and designed for fedora users…it uses calls to their package manager called “YUM”…after some searching on rpmfind.net I was able to locate a recent copy of yum for my opensuse x86_64 system and get it installed…and it appears to be working fine. I also made some tweaks to my squid to ensure video’s and pics etc would get cached as long as possible while still getting current data. I’ll past some of the pertinent parts of my squid.conf to show what I am talking about. A dnscache such as dnsmasq with the cache entries set to 1000+ entries should help alot too especially if you have it using the opendns servers instead of your isp’s.
The adzap plugin came from: Ad Zapping With Squid
The intelligentmirror plugin came from: https://fedorahosted.org/intelligentmirror/wiki/IntelligentMirror I installed the noarch rpm and then added the entries to config file after installing YUM which isn’t listed as a dependency but is necessary to provide needed python modules.
I changed the refresh patterns in squid to keep the graphics/files longer…the refresh patterns for that are:
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 10080 90% 43200 override-expire ignore-no-cache ignore-private
refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90% 432000 override-expire ignore-no-cache ignore-priv$
refresh_pattern -i \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff)$ 10080 90% 43200 override-expire ignore-no-cache ig$
refresh_pattern -i \.index.(html|htm)$ 0 40% 10080
refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320
refresh_pattern . 0 40% 40320 reload-into-ims
You can add “ignore-no-store” to the lines that have “ignore-no-cache” if running squid 3.0 or higher. 2.7 doesn’t support that directive. The patterns used came from this article on linux.com:
Linux.com :: Speed up your Internet access using Squid’s refresh patterns
The line to call wrapzap (which calls adzap) and the intelligent mirror ones are at the bottom of my config.
# windows_ipaddrchangemonitor on
redirect_program /usr/local/adzap/scripts/wrapzap
# --BEGIN-- squid config for intelligentmirror #
storeurl_rewrite_program /usr/bin/python /etc/squid/intelligentmirror/intelligentmirror.py
storeurl_rewrite_children 15
acl store_rewrite_list urlpath_regex -i .rpm$
# acl store_rewrite_list urlpath_regex -i .deb$
storeurl_access allow store_rewrite_list
storeurl_access deny all
# --END-- squid config for intelligentmirror #
The program supports .deb’s…spose it could easily support other formats but I commented the .deb out since I only needed rpm’s…when the intelligent mirror intercepts a rpm download it’s log file shows:
2009-10-17 00:13:57,283 INFO NEW_URL RPM gtk2-metatheme-sonar-11.2.5-1.1.noarch.rpm http://192.168.2.2/rpm/gtk2-metatheme-sonar-11.2.5-1.1.noarch.rpm
2009-10-17 00:14:05,355 INFO NEW_URL RPM gftp-lang-2.0.19-1.20.noarch.rpm http://192.168.2.2/rpm/gftp-lang-2.0.19-1.20.noarch.rpm
2009-10-17 00:14:09,394 INFO NEW_URL RPM gftp-lang-2.0.19-1.20.noarch.rpm http://192.168.2.2/rpm/gftp-lang-2.0.19-1.20.noarch.rpm
2009-10-17 00:14:09,395 INFO NEW_URL RPM gftp-lang-2.0.19-1.20.noarch.rpm http://192.168.2.2/rpm/gftp-lang-2.0.19-1.20.noarch.rpm
2009-10-17 00:15:23,762 INFO NEW_URL RPM command-not-found-0.1.0-33.1.noarch.rpm http://192.168.2.2/rpm/command-not-found-0.1.0-33.1.noarch.rpm
2009-10-17 00:15:27,943 INFO NEW_URL RPM command-not-found-0.1.0-33.1.noarch.rpm http://192.168.2.2/rpm/command-not-found-0.1.0-33.1.noarch.rpm
2009-10-17 00:15:35,496 INFO NEW_URL RPM zenity-2.28.0-1.2.i586.rpm http://192.168.2.2/rpm/zenity-2.28.0-1.2.i586.rpm
2009-10-17 00:15:38,573 INFO NEW_URL RPM zenity-2.28.0-1.2.i586.rpm http://192.168.2.2/rpm/zenity-2.28.0-1.2.i586.rpm
You can see where it’s rewriting them to show my local 192.168.2.2 pc. This makes keeping multiple pc’s synced up alot easier. The actual version of Yum I installed is ftp://ftp5.gwdg.de/pub/opensuse/repositories/home:/cthiel1/Factory/x86_64/yum-3.2.20-2.2.x86_64.rpm
I’ll try to attach my squid.conf…it’s not tiny and I’m gonna leave the comments in there so you know what you are changing…the transparent proxying was relatively easy…I added the word “transparent” after the 3128 port and setup to iptables rules on router/pc:
iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
iptables -t nat -A PREROUTING -i eth0 -p udp -m udp --dport 80 -j REDIRECT --to-ports 3128
I also added some lines to my dd-wrt router which acts as my local dns server to FORCE all pc’s on the network to use the router for dns…and it in return uses opendns…this helps alot…those lines are:
iptables -t nat -A PREROUTING -p udp -i br0 --dport 53 -j DNAT --to $(nvram get lan_ipaddr)
iptables -t nat -A PREROUTING -p tcp -i br0 --dport 53 -j DNAT --to $(nvram get lan_ipaddr)
You could easily change the nvram command to just be 192.168.2.1 which is my routers ip address.
I will try to help if someone has questions but I may refer you to the same tutorials I read on google such as how to setup the transparent proxy…etc. Hope this helps someone and I’d love input/comments.
My actual squid.conf is available at http://famewolf.tripod.com/squid.conf because it was to large to attach to the msg.