Proxy Server : SQUID - ALL-in-One Tutorials and Other Advanced Stuffs
Installation & Configuration
Web Proxy
A proxy is an intermediary/middle-agent between computer/computers & other resources, mostly internet. It seeks requests from client & transfer them to internet.
Benefits of a Web Proxy
- It can be used to accelerate the internet as a proxy can build up a cache of frequently used websites, which makes it easier & faster to load up after,
- Can be used to block/allow websites as required,
- also can be used to bypass another web proxy . For example in many organizations Social networking websites like Facebook, Twitter , Youtube etc are not allowed. So a web proxy can be used to bypass those restrictions & provide access to restricted websites.
Squid proxy server
Its a caching proxy server which supports HTTP, HTTPS, FTP . It can be used as an accelerating server, thereby decreasing response time & reducing bandwidth. It can also be used for the purpose of Web filtering due to availability of extensive access controls.
And we will be exploring web filtering part in this tutorial.
Scenario Setup
Firstly, to test or create a squid proxy setup, we will need a squid server & a client machine.
Squid server Client’s Machine
OS : Centos/RHEL 6 or 7 OS: Centos/RHEL 6 or 7
Hostname : server.test.com Hostname: client1.test.com
IP Address :192.168.1.100 IP Address : 192.168.1.101
Important
Configuration file /etc/squid/squid.conf
Default port 3128
Installation
In order to install, use following command
yum install squid -y
Configuration
We need to create an ACL rule (Access Control List), which is the list or rule with list of access control entries.Some acl rules are already written in configuration file by default in the configuration file,
acl localhost src 127.0.0.1/32
http_access allow localhost (some lines below the above line)
http_access allow localhost (some lines below the above line)
So, this is what an acl rule look like. Lets see what these means,
firstly,acl this is declaring that a new acl is starting
then,localhost is the name of acl created
src is used in case acl is for local Ipadress , srcdomain is used for declaring Localdomain, dst for public IP & dstdomain for publlic domain name
and lastly,127.0.01/32 declares the IP Address on which the acl is to be applied, in this case its localhost or 127.0.0.1
Next line i.e. http_access allow localhost, means
http_access will initiate an action based on next word
allow/deny will either allow or deny access
and,localhost again is the name of acl as declared above.
So, basically that how we create a ACL/rule in squid proxy server.
Now, lets restart our server (with default config file) & configure the client machine to see if proxy is working properly.
service squid restart
chkconfig squid on
chkconfig squid on
Note Its always wise to have a backup of original configuration file when starting to make changes. So, create a backup a backup of before starting.
Configuration on Client Side
Open Firefox Browser &
- Open Edit menu —> Preferences —> Advanced —-> Settings
- Check the box ‘ Manual proxy configuration’ & enter IP Address & Port Number of squid proxy server.
In our case its 192.168.1.100 & 3128.
- Click OK
& that’s all we need to configure on Client’s side.
Then we check out if its works. Open a website (example Facebook.com), if proxy server is working properly you will be greeted with an error ‘ Access Denied’. That’s because by default internet access is denied for all in server.
Now, lets check logs in server, to see if a request was received by proxy server or not,
tail -f /var/log/squid/access.log
and it should show you all the received requests from client to server.
Restricting access to websites
In order to restrict access to a website, open configuration file & then create a new acl
acl blacksite .facebook.com
and deny access to the acl
Note Also set “http_access deny all” to “http_access allow all “, otherwise we wont be able to access internet.
Now, restart your squid proxy server to apply changes or we can also use squid -k reconfigure to implement changes to server without restarting the server.
then, we will access client’s machine and open Facebook but you wont be able to access it at all. As for other websites you can access them just fine.
Blocking Multiple websites
Firstly, we will create a file named blacksites (or bad-domains or whatever )
vi /etc/squid/blacksites
and add the websites we need blocked & save the file
.facebook.com
.youtube.com
.twitter.com
.youtube.com
.twitter.com
Now, open main configuration file
vi /etc/squid/squid.conf
and create a new acl
acl blacksites dstdomain /etc/squid/blacksites
then, we deny access to the created acl
http_access deny blacksites
lastly, restart proxy server to apply changes.
service squid restart
Note you can also use squid -k reconfigure to apply changes to server without actually restarting the server.
Time based acl
Sometimes, we might require access to a blocked website for a certain period of time or we might need to block certain websites for certain time. This can be achieved using a time based acl
Firstly, open configuration file
vi /etc/squid/squid.conf
then create a new acl and allow access to the acl
acl timebased time MTW 10:30-11:30
http_access allow blacksites
http_access allow blacksites
lastly, restart your server to implement changes. & we now have access of blocked sites on Monday, Tuesday & Wednesday between 10:30AM to 11:30AM .
Enabling cache to speed up browsing
So, by enabling cache in our server we can speed up our browsing speed for frequently visited pages.
By adding just one line in our configuration file, we can enable cache.
To enable cache , open configuration file
vi /etc/squid/squid.conf
and add following line to bottom of the file
cache_dir ufs /var/cache/squid 2000 16 256
where ufs is squid storage format,
/var/cache/squid is path for cache storage,
2000 is size in MB can be used for cache,
and, 16 is number of 1st level sub-directories & 256 is 2nd level sub directories in cache folder.
SARG ( Squid Analysis Report Generator )- Installation & Configuration
We will be discussing SARG (Squid Analysis Report Generator).
It is an Open-Source tool, which helps us analyze Squid Proxy logs & generates reports in HTML format with all the information from logs presented in nice & easy to understand format.
& It gives information about User’s IP addresses , total & individually used bandwidth etc with access to Daily, Weekly & Monthly reports.
Installation
The process for installing sarg on Centos/Redhat is a bit complicated, as it needs to be compiled from source. To do that, firstly we need to install required packages to download & compile the package
yum install -y gcc gd gd-devel make perl-GD wget httpd
Secondly, download ppackage from the link mentioned in below
wget http://sourceforge.net/projects/sarg/files/sarg/sarg-2.3.10/sarg-2.3.10.tar.gz
now, we will extract the package & will than compile the package
tar -xvzf sarg-2.3.10.tar.gz
cd sarg-2.3.10
./configure
make
make install
cd sarg-2.3.10
./configure
make
make install
Configuration
Now that’s the installation is complete, we will configure it as per our needs by making changes in configuration file
vi /usr/local/etc/sarg.conf
Firstly, uncomment the line starting with access_log & add path for squid access log. Next, provide output directory for reports next to line starting with output_dir & also select your desired time format, change line with date_format
# TAG: access_log file
# Where is the access.log file
#
#
access_log /var/log/squid/access.log
Add output directory
# TAG: output_dir
# The reports will be saved in that directory
#
#
output_dir /var/www/html/squid
Set the correct date format
# TAG: date_format
# Date format in reports: e (European=dd/mm/yy), u (American=mm/dd/yy), w (Weekly=yy.ww)
#
date_format e
# Where is the access.log file
#
#
access_log /var/log/squid/access.log
Add output directory
# TAG: output_dir
# The reports will be saved in that directory
#
#
output_dir /var/www/html/squid
Set the correct date format
# TAG: date_format
# Date format in reports: e (European=dd/mm/yy), u (American=mm/dd/yy), w (Weekly=yy.ww)
#
date_format e
& lastly , set overwrite report to yes
# # TAG: overwrite_report yes|no
# yes – if report date already exist then will be overwritten.
# no – if report date already exist then will be renamed to filename.n, filename.n+1
#
overwrite_report yes
# yes – if report date already exist then will be overwritten.
# no – if report date already exist then will be renamed to filename.n, filename.n+1
#
overwrite_report yes
Generating report
To create squid analysis report, we have to enter following command
sarg -x
Note: It may take a while depending on number of users accessing squid proxy.
Accessing report
To access the report, enter below mentioned URL in web-browser
http://IP-Address of server/squid
Now, we have all the squid analyzed logs in nice, sorted &easy to understand format
Note : you can also create a cron–job to schedule a report being generated automatically at the time of your choosing.
For Example
* */4 * * * /usr/local/bin/sarg -x
This will generate a report every 4th hour.
How to setup SQUID AUTHENTICATION
In this tutorial, we are going to discuss to setup squid authentication . Before we start we must know that SQUID AUTHENTICATION CAN NOT BE USED WHEN CONFIGURED AS TRANSPARENT PROXY. Now that we got it out of hte way, let’s start our configurations for setting up squid with authentication ,
Pre-requisites for Squid Authentication
We need to have both squid & http packges installed on our sytem. if not installed, install the packages using the following commands,
$ sudo yum install httpd
$ sudo yum install squid
To detailed squid installation, refer to the articles mentioned above.
Configuration
We are going to use module called ‘ncsa_auth’ for squid authetication. It’s located at ‘/usr/lib/squid/ncsa_auth’ for 32 bit systems & for 64 bit system, it’s located at ‘/usr/lib64/squid’ directory.
So firstly, we need to makes changes in squid’s main configuration file i.e. ‘/etc/squid/squid.conf’, so open the file using vim
$ sudo vim /etc/squid/squid.conf
firstly we will assign the squid authentication method, so add the following line in squid.conf
auth_param basic program /usr/lib64/squid/ncsa_auth /etc/squid/users_passwd
auth_param basic realm proxy
here, /etc/squid/users_passwd is the files with user information. Next we will setup an acl named ‘auth_users’ for the authentication,
acl auth_users proxy_auth REQUIRED
Now we will apply the created acl
http_access allow auth_users
http_access deny all
Now save the file & exit but make sure that all these acl should be entered above all other acls otherwise they might not work.
Now we only need to create users for authentication. To create the password table, we need to execute the following command,
$ sudo htpasswd -c /etc/squid/users_passwd squid
here ‘-c’ option is used to create the file & will not be used for adding other users, ‘squid’ is the name user. Now we need to restart the squid server to implement all the changes.
$ sudo service squid restart
We can also use PAM, RADIUS or DIGEST for setting up squid authentication, but for this tutorial it’s NCSA_AUTH, maybe in some future tutorials we will discuss those.
Squid Transparent proxy server : How to configure
Squid as transparent proxy acts as a gateway between internet and users. It redirects all the internet traffic from port 80 to squid proxy’s port i.e. 3128. So now let’s start with the setting squid as transparent proxy…
Installation
Firstly, we need to install squid proxy server on the system. To install it, execute
$ sudo yum install squid -y
Configuring squid
Next we need to enable IP Packet Forwarding on the machine, to do this
$ sudo vim /etc/sysctl.conf
then change the following parameter to ‘1’, i.e.
net.ipv4.ip_forward = 1
Save file & exit. Now execute the following command to implement the changes made,
$ sudo sysctl -p
Next, we will configure the squid proxy using it’s main configuration file i.e. ‘/etc/squid/squid.conf’,
$ sudo vim /etc/squid/squid.conf
& make changes as follows to the options mentioned,
http_access allow all
http_port 3128 intercept
visible_hostname squid.proxy
Now save the file & exit. Next to implement the changes restart the squid service,
$ sudo service squid restart
$ chkconfig squid on
Configuring firewall rules
All that remains in the configuration for setting up squid transparent proxy is firewall rules configuration. Firewalld rules for RHEL/CentOS 7 are ,
$ sudo firewall-cmd –permanent –zone=public –add-forward- port=port=80:proto=tcp:toport=3128:toaddr=192.168.1.10
$ sudo firewall-cmd –permanent –zone=public –add-port=3128/tcp
$ sudo firewall-cmd –permanent –add-masquerade
$ sudo firewall-cmd –reload
here, 192.168.1.10 is the LAN IP address of the squid proxy server.
For RHEL/CentOS 6, the Iptables rules are
$ sudo iptables -t nat -A PREROUTING -i eth0 -p tcp –dport 80 -j DNAT –to 192.168.1.10:3128
$ sudo iptables -t nat -A PREROUTING -i eth1 -p tcp –dport 80 -j REDIRECT –to-port 3128
$ sudo iptables –t nat -A POSTROUTING –out-interface eth1 -j MASQUERADE
After the changes have been made to firewall rules, our server is now ready to work as a squid transparent proxy. All we have to do test this, is to change the gateway of the client machine to the IP address of squid server i.e. 192.168.1.10. When we access any website from client machine, it will first arrive at squid proxy server on port 80 & will then be redirected to port 3128, then after analysing the ACLs, traffic will be forward to WAN or internet.
Install SquidGuard with Squid proxy server : Complete guide
SquidGuard is a free and open-source application, which is basically used as a URL redirectore software, which we can use to managing access to websites. Squidguard uses blacklists to control access to websites & define the website to which URL has to be redirected. We have option to create a custom blacklist or can use one of many available on the internet. SquidGuard can be installed on Linux as well Unix.
Pre-requisites
Package to install SquidGuard are not available with default Centos/RHEL repos, we need to enable the EPEL repository on our system to install squidguard. Install EPEL repository the following command, applicable to you,
RHEL/CentOS 7
# rpm -Uvh https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm
RHEL/CentOS 6 (64 Bit)
# rpm -Uvh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
RHEL/CentOS 6 (32 Bit)
# rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
Install SquidGuard
We first need to install squid on our system. If you have not done it already, install squid with the following command,
# yum install squid
Now install squidguard with the following command,
# yum install squidGuard
Note:- Mind the uppercase ‘G’ in squidGuard.
So we now squid with squidguard installed on our server. Now let’s complete the configuration,
Configuring Squidguard
We will firstly create a folder for keeping a blaclikst named ‘test’ in folder ‘/var/squidGurad/test’.
# mkdir -p /var/squidGuard/test
In the folder test, we will create two files, one by the name ‘domains’ & other by the name ‘urls’ with the list on domains and URLs we want to redirect/block,
# cd /var/squidGuard/test
# vi domains
facebook.com
twitter.com
youtube.com
# vi urls
www.facebook.com
www.twitter.com
www.youtube.com
We have our blacklist ready to use. We will now configure the squidGuard & will create ACL for the blacklist. Open the squidguard configuration file & make the following entries,
# vi /etc/squid/squidGuard.conf
dbhome /var/squidGuard/db
logdir /var/log/squidGuard
dest deny {
domainlist test/domains
urllist test/urls
}
acl {
default {
pass !test all
redirect https://google.com
}
}
Above file has some default configurations, you can either delete the contents of the file or can backup the file to another location or by another name. We will now make a change to squid configuration file as well. We will mention the URL rediretion setting to squid configuration file,
# vi /etc/squid/squid.conf
& add the following line to the bottom of the file,
url_rewrite_program /usr/bin/squidGuard -c /etc/squid/squidGuard.conf
Save the file & exit. Next we need to make sure that all the squidguard related files & folders have ownership for user squid,
# chown -R squid:squid /var/squidGuard
# chown -R squid:squid /var/log/squidGuard
# chown squid:squid /etc/squid/squidGuard.conf
We are now ready to create the db files for the blacklist, execute the following command to complete the action,
# squidGuard -b -d -C all
Once complete, either restart the squid server or reconfigure it with the following command,
# squid -k reconfigure
That’s it guys, we now have our fully operational SquidGuard with squid proxy server. All we have to do is to make the proxy settings in the client machine & we can utilize the squidGuard redirection capablities.
Configuring a Blacklist
If you are trying to use a blacklist downloaded from internet like Shalla’s Blacklist, you can follow the example below. First download the blacklist,
# wget http://www.shallalist.de/Downloads/shallalist.tar.gz
& extract the blacklist,
# tar -xvzf shallist.tar.gz -C /var/squidGuard/
Now there will be several folders (based on the URL category ) in the folder, choose the categories that you need to block & create the ACL for that in squidGuard.conf. For example,
dbhome /var/squidGuard/db
logdir /var/log/squidGuard
dest downloads {
domainlist downloads/domains
urllist downloads/urls
}
acl {
default {
pass !downloads all
redirect https://google.com
}
}
Update the squidguard DB & reconfigure the squid server,
# squidGuard -b -d -C all
# squid -k reconfigure
With this we end our tutorial on how to install SquidGuard & use it with squid proxy server.
Some pretty useful SQUID tips & tricks
Squid Tips & tricks
=====================================================================================
Setting up squid with a customized port
Default port number for Squid proxy server is 3128 but we can change it any other port as per our requirement. For example, if we want to change the default port from 3128 to 8080, we can do so by making any entry for same in squid configuration file i.e. ‘/etc/squid/squid.conf‘
Open squid.conf,
$ sudo vim /etc/squid/squid.conf
& search for ‘http_port 3128’ & change it to
http_port 8080
Save file & exit. Than restart the squid service to implement the changes.
$ sudo systemctl restart squid
We can also use
$ sudo squid -k reconfigure
to implement changes, without restarting the squid service.
Limit download size of the files
We can also impose a download limit based on file size for squid proxy users. We need to provide ‘reply_max_body_size’ directive in squid.conf file to impose a file size limit.
Open squid.conf
$ sudo vim /etc/squid/squid.conf
& make the following entry in the bottom of the file
reply_body_max_size 5120 KB all
This will impose a limit of max 5Mb download for a single file for all the squid users. We can also impose limit based on user,
reply_body_max_size 5120 KB user1 user2
or based on group,
reply_body_max_size 5120 KB user1 group2
After making changes to file save it & restart or reconfigure squid to implement the changes.
Limit upload size of the files
To impose limit of size of file that can be uploaded, we use “request_body_max_size”. Make the following entry in squid.conf to impose limit on file uploads,
$ sudo vim /etc/squid/squid.conf
request_body_max_size 100kb
Save file & restart/reconfigure the squid server to implement changes. We can also impose upload limits based on users & groups, same as we imposed limits on file download.
Allow Squid proxy to cache all requests
To cache all the requests that are made in squid server, add the following directive in squid.conf file,
$ vim /etc/squid/squid.conf
cache allow all
Save file & reconfigure the squid server to implement the changes.
Disable caching of some websites
To disable cache only on some website, we will first create an ACL with websites & than disable the caching for that ACL. Make the following entries on squid.conf file,
$ vim /etc/squid/squid.conf
acl NOcache_websites dstdomain www.linuxtechlab.com www.msn.com
no_cache deny Nocache_websites
Save file & restart/reconfigure the squid server to implement changes.
Deny cache requests for some file extensions
To disable caching of files with certain extension, add the following lines in squid.conf file.
$ sudo vim /etc/squid/squid.conf
hierarchy_stoplsit .mp3 ?
hierarchy_stoplist .xls ?
acl ext1 urlpath_regex \.mp3 \?
acl ext2 urlpath_regex \.xls \?
no_cache deny ext1
no_cache deny ext2
Here, we have blocked caching for MP3 & XLS files. Save file & restart/reconfigure the squid server to implement changes.
Note:- To remove existing cached file, execute following command from terminal,
$ sudo rm -rf /var/spool/squid/*
Deny all cache requests
To deny all cache requests made to squid server, add the following directive in squid.conf file
$ vim /etc/squid/squid.conf
cache deny all
Save file & reconfigure the squid server to implement the changes.
