Proxy Server : SQUID - ALL-in-One Tutorials and Other Advanced Stuffs

Proxy Server : SQUID - ALL-in-One Tutorials and Other Advanced Stuffs

Installation & Configuration

Web Proxy

A proxy is an intermediary/middle-agent between computer/computers & other resources, mostly internet. It seeks requests from client & transfer them to internet.

Benefits of a Web Proxy

• It can be used to accelerate the internet as a proxy can build up a cache of frequently used websites, which makes it easier & faster to load up after,
• Can be used to block/allow websites as required,
• also can be used to bypass another web proxy . For example in many organizations Social networking websites like Facebook, Twitter , Youtube etc are not allowed. So a web proxy can be used to bypass those restrictions & provide access to restricted websites.

Squid proxy server

Its a caching proxy server which supports HTTP, HTTPS, FTP . It can be used as an accelerating server, thereby decreasing response time & reducing bandwidth. It can also be used for the purpose of Web filtering due to availability of extensive access controls.

And we will be exploring web filtering part in this tutorial.

Scenario Setup

Firstly, to test or create a squid proxy setup, we will need a squid server & a client machine.

Squid server                                                              Client’s Machine

OS : Centos/RHEL 6 or 7                                       OS: Centos/RHEL 6 or 7

Hostname : server.test.com                                   Hostname: client1.test.com

IP Address :192.168.1.100                                      IP Address : 192.168.1.101

Important

Configuration file       /etc/squid/squid.conf

Default port                 3128

Installation

In order to install, use following command

yum install squid -y

Configuration

We need to create an ACL rule (Access Control List), which is the list or rule with list of access control entries.Some acl rules are already written in configuration file by default in the configuration file,

acl localhost src 127.0.0.1/32
http_access allow localhost                                               (some lines below the above line)

So, this is what an acl rule look like. Lets see what these means,

firstly,acl this is declaring that a new acl is starting

then,localhost is the name of acl created

src is used in case acl is for local Ipadress , srcdomain is used for declaring Localdomain, dst for public IP & dstdomain for publlic domain name

and lastly,127.0.01/32 declares the IP Address on which the acl is to be applied, in this case its localhost or 127.0.0.1

Next line i.e. http_access allow localhost, means

http_access will initiate an action based on next word

allow/deny will either allow or deny access

and,localhost again is the name of acl as declared above.

So, basically that how we create a ACL/rule in squid proxy server.

Now, lets restart our server (with default config file) & configure the client machine to see if proxy is working properly.

service squid restart
chkconfig squid on

Note  Its always wise to have a backup of original configuration file when starting to make changes. So, create a backup a backup of before starting.

 Configuration on Client Side

Open Firefox Browser &

• Open Edit menu —> Preferences —> Advanced —-> Settings
• Check the box ‘ Manual proxy configuration’ & enter IP Address & Port Number of squid proxy server.

In our case its 192.168.1.100 & 3128.

• Click OK

& that’s all we need to configure on Client’s side.

Then we check out if its works. Open a website (example Facebook.com), if proxy server is working properly you will be greeted with an error ‘ Access Denied’. That’s because by default internet access is denied for all in server.

Now, lets check logs in server, to see if a request was received by proxy server or not,

tail -f /var/log/squid/access.log

and it should show you all the received requests from client to server.

Restricting access to websites

In order to restrict access to a website, open configuration file & then create a new acl

acl blacksite .facebook.com

and deny access to the acl

Note Also set “http_access deny all” to “http_access allow all “, otherwise we wont be able to access internet.

Now, restart your squid proxy server to apply changes or we can also use squid -k reconfigure to implement changes to server without restarting the server.

then, we will access client’s machine and open Facebook but you wont be able to access it at all. As for other websites you can access them just fine.

Blocking Multiple websites

Firstly, we will create a file named blacksites (or bad-domains or whatever )

vi /etc/squid/blacksites

and add the websites we need blocked & save the file

.facebook.com
.youtube.com
.twitter.com

Now, open main configuration file

vi /etc/squid/squid.conf

and create a new acl

acl blacksites dstdomain /etc/squid/blacksites

then, we deny access to the created acl

http_access deny blacksites

lastly, restart proxy server to apply changes.

service squid restart

Note you can also use squid -k reconfigure to apply changes to server without actually restarting the server.

Time based acl

Sometimes, we might require access to a blocked website for a certain period of time or we might need to block certain websites for certain time. This can be achieved using a time based acl

Firstly, open configuration file

vi /etc/squid/squid.conf

then create a new acl and allow access to the acl

acl timebased time MTW 10:30-11:30
http_access allow blacksites

lastly, restart your server to implement changes. & we now have access of blocked sites on Monday, Tuesday & Wednesday between 10:30AM to 11:30AM .

Enabling cache to speed up browsing

So, by enabling cache in our server we can speed up our browsing speed for frequently visited pages.

By adding just one line in our configuration file, we can enable cache.

To enable cache , open configuration file

vi /etc/squid/squid.conf

and add following line to bottom of the file

cache_dir ufs /var/cache/squid 2000 16 256

where ufs is squid storage format,

/var/cache/squid is path for cache storage,

2000 is size in MB can be used for cache,

 and, 16 is number of 1^st level sub-directories & 256 is 2^nd level sub directories in cache folder.

SARG ( Squid Analysis Report Generator )- Installation & Configuration

We will be discussing SARG (Squid Analysis Report Generator).

It is an Open-Source tool, which helps us analyze Squid Proxy logs & generates reports in HTML format with all the information from logs presented in nice & easy to understand format.

& It gives information about User’s IP addresses , total & individually used bandwidth etc with access to Daily, Weekly & Monthly reports.

Installation

The process for installing sarg on Centos/Redhat is a bit complicated, as it needs to be compiled from source. To do that, firstly we need to install required packages to download & compile the package

yum install -y gcc gd gd-devel make perl-GD wget httpd

Secondly, download ppackage from the link mentioned in below

wget http://sourceforge.net/projects/sarg/files/sarg/sarg-2.3.10/sarg-2.3.10.tar.gz

now, we will extract the package & will than compile the package

tar -xvzf sarg-2.3.10.tar.gz
cd sarg-2.3.10
./configure
make
make install

Configuration

Now that’s the installation is complete, we will configure it as per our needs by making changes in configuration file

vi /usr/local/etc/sarg.conf

Firstly, uncomment the line starting with access_log & add path for squid access log. Next, provide output directory for reports next to line starting with output_dir & also select your desired time format, change  line with date_format

# TAG: access_log file
# Where is the access.log file
#
#
access_log /var/log/squid/access.log
Add output directory
# TAG: output_dir
# The reports will be saved in that directory
#
#
output_dir /var/www/html/squid
Set the correct date format
# TAG: date_format
# Date format in reports: e (European=dd/mm/yy), u (American=mm/dd/yy), w (Weekly=yy.ww)
#
date_format e

& lastly , set overwrite report to yes

# # TAG: overwrite_report yes|no
# yes – if report date already exist then will be overwritten.
# no – if report date already exist then will be renamed to filename.n, filename.n+1
#
overwrite_report yes

Generating report

To create squid analysis report, we have to enter following command

sarg -x

Note: It may take a while depending on number of users accessing squid proxy.

Accessing report

To access the report, enter below mentioned URL in web-browser

http://IP-Address of server/squid

Now, we have all the squid analyzed logs in nice, sorted &easy to understand format

Note : you can also create a cron–job to schedule a report being generated automatically at the time of your choosing.

For Example

 * */4 * * * /usr/local/bin/sarg -x

This will generate a report every 4^th hour.

How to setup SQUID AUTHENTICATION

In this tutorial, we are going to discuss to setup squid  authentication . Before we start we must know that SQUID AUTHENTICATION CAN NOT BE USED WHEN CONFIGURED AS TRANSPARENT PROXY. Now that we got it out of hte way, let’s start our configurations for setting up squid with authentication ,

Pre-requisites for Squid Authentication

We need to have both squid & http packges installed on our sytem. if not installed, install the packages using the following commands,

$ sudo yum install httpd

$ sudo yum install squid

To detailed squid installation, refer to the articles mentioned above.

Configuration

We are going to use module called ‘ncsa_auth’ for squid authetication. It’s located at ‘/usr/lib/squid/ncsa_auth’ for 32 bit systems & for 64 bit system, it’s located at ‘/usr/lib64/squid’ directory.

So firstly, we need to makes changes in squid’s main configuration file i.e. ‘/etc/squid/squid.conf’, so open the file using vim

$ sudo vim /etc/squid/squid.conf

firstly we will assign the squid authentication method, so add the following line in squid.conf

auth_param basic program /usr/lib64/squid/ncsa_auth /etc/squid/users_passwd

auth_param basic realm proxy

here, /etc/squid/users_passwd is the files with user information. Next we will setup an acl named ‘auth_users’ for the authentication,

acl auth_users proxy_auth REQUIRED

Now we will apply the created acl

http_access allow auth_users

http_access deny all

Now save the file & exit but make sure that all these acl should be entered above all other acls otherwise they might not work.

Now we only need to create users for authentication. To create the password table, we need to execute the following command,

$ sudo htpasswd -c /etc/squid/users_passwd squid

here ‘-c’ option is used to create the file & will not be used for adding other users, ‘squid’ is the name user. Now we need to restart the squid server to implement all the changes.

$ sudo service squid restart

We can also use PAM, RADIUS or DIGEST for setting up squid authentication, but for this tutorial it’s NCSA_AUTH, maybe in some future tutorials we will discuss those.

Squid Transparent proxy server : How to configure

 Squid as transparent proxy acts as a gateway between internet and users. It redirects all the internet traffic from port 80 to squid proxy’s port i.e. 3128. So now let’s start with the setting squid as transparent proxy…

Installation

Firstly, we need to install squid proxy server on the system. To install it, execute

$ sudo yum install squid -y

Configuring squid

Next we need to enable IP Packet Forwarding on the machine, to do this

$ sudo vim /etc/sysctl.conf

then change the following parameter to ‘1’, i.e.

net.ipv4.ip_forward = 1

Save file & exit. Now execute the following command to implement the changes made,

$ sudo sysctl -p

Next, we will configure the squid proxy using it’s main configuration file i.e. ‘/etc/squid/squid.conf’,

$ sudo vim /etc/squid/squid.conf

& make changes as follows to the options mentioned,

http_access allow all

http_port 3128 intercept

visible_hostname squid.proxy

Now save the file & exit. Next to implement the changes restart the squid service,

$ sudo service squid restart

$ chkconfig squid on

Configuring firewall rules

All that remains in the configuration for setting up squid transparent proxy is firewall rules configuration. Firewalld rules for RHEL/CentOS 7 are ,

$ sudo firewall-cmd –permanent –zone=public –add-forward- port=port=80:proto=tcp:toport=3128:toaddr=192.168.1.10

$ sudo firewall-cmd –permanent –zone=public –add-port=3128/tcp

$ sudo firewall-cmd –permanent –add-masquerade

$ sudo firewall-cmd –reload

here, 192.168.1.10 is the LAN IP address of the squid proxy server.

For RHEL/CentOS 6, the Iptables rules are

$ sudo iptables -t nat -A PREROUTING -i eth0 -p tcp –dport 80 -j DNAT –to 192.168.1.10:3128

$ sudo iptables -t nat -A PREROUTING -i eth1 -p tcp –dport 80 -j REDIRECT –to-port 3128

$ sudo iptables –t nat -A POSTROUTING –out-interface eth1 -j MASQUERADE

After the changes have been made to firewall rules, our server is now ready to work as a squid transparent proxy. All we have to do test this, is to change the gateway of the client machine to the IP address of squid server i.e. 192.168.1.10. When we access any website from client machine, it will first arrive at squid proxy server on port 80 & will then be redirected to port 3128, then after analysing the ACLs, traffic will be forward to WAN or internet.

 

Install SquidGuard with Squid proxy server : Complete guide

SquidGuard is a free and open-source application, which is basically used as a URL redirectore software, which we can use to managing access to websites. Squidguard uses blacklists to control access to websites & define the website to which URL has to be redirected. We have option to create a custom blacklist or can use one of many available on the internet. SquidGuard can be installed on Linux as well Unix. 

Pre-requisites

Package to install SquidGuard are not available with default Centos/RHEL repos, we need to enable the EPEL repository on our system to install squidguard. Install EPEL repository the following command, applicable to you,

RHEL/CentOS 7

# rpm -Uvh https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm

RHEL/CentOS 6 (64 Bit)

# rpm -Uvh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

RHEL/CentOS 6 (32 Bit)

# rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm

Install  SquidGuard

We first need to install squid on our system. If you have not done it already, install squid with the following command,

# yum install squid

Now install squidguard with the following command,

# yum install squidGuard

Note:- Mind the uppercase ‘G’ in squidGuard.

So we now squid with squidguard installed on our server. Now let’s complete the configuration,

Configuring Squidguard

We will firstly create a folder for keeping a blaclikst named ‘test’ in folder ‘/var/squidGurad/test’.

# mkdir -p /var/squidGuard/test

In the folder test, we will create two files, one by the name ‘domains’ & other by the name ‘urls’ with the list on domains and URLs we want to redirect/block,

# cd /var/squidGuard/test

# vi domains

facebook.com

twitter.com

youtube.com

# vi urls

www.facebook.com

www.twitter.com

www.youtube.com

We have our blacklist ready to use. We will now configure the squidGuard & will create ACL for the blacklist. Open the squidguard configuration file & make the following entries,

# vi /etc/squid/squidGuard.conf

dbhome /var/squidGuard/db

logdir /var/log/squidGuard

dest deny {

                 domainlist test/domains

                 urllist test/urls

                 }

acl {

             default {

                           pass !test all

                           redirect https://google.com

                          }

}

Above file has some default configurations, you can either delete the contents of the file or can backup the file to another location or by another name. We will now make a change to squid configuration file as well. We will mention the URL rediretion setting to squid configuration file,

# vi /etc/squid/squid.conf

& add the following line to the bottom of the file,

url_rewrite_program /usr/bin/squidGuard -c /etc/squid/squidGuard.conf

Save the file & exit. Next we need to make sure that all the squidguard related files & folders have ownership for user squid,

# chown -R squid:squid /var/squidGuard

# chown -R squid:squid /var/log/squidGuard

# chown squid:squid /etc/squid/squidGuard.conf

We are now ready to create the db files for the blacklist, execute the following command to complete the action,

# squidGuard -b -d -C all

Once complete, either restart the squid server or reconfigure it with the following command,

# squid -k reconfigure

That’s it guys, we now have our fully operational SquidGuard with squid proxy server. All we have to do is to make the proxy settings in the client machine & we can utilize the squidGuard redirection capablities.

Configuring a Blacklist

If you are trying to use a blacklist downloaded from internet like Shalla’s Blacklist, you can follow the example below. First download the blacklist,

# wget http://www.shallalist.de/Downloads/shallalist.tar.gz

& extract the blacklist,

# tar -xvzf shallist.tar.gz -C /var/squidGuard/

Now there will be several folders (based on the URL category ) in the folder, choose the categories that you need to block & create the ACL for that in squidGuard.conf. For example,

dbhome /var/squidGuard/db

logdir /var/log/squidGuard

dest downloads {

                           domainlist downloads/domains

                           urllist downloads/urls

                          }

acl {

         default {

                        pass !downloads all

                        redirect https://google.com

                      }

}

Update the squidguard DB & reconfigure the squid server,

# squidGuard -b -d -C all

# squid -k reconfigure

With this we end our tutorial on how to install SquidGuard & use it with squid proxy server. 

 

 

Some pretty useful SQUID tips & tricks

Squid Tips & tricks

=====================================================================================

Setting up squid with a customized port

Default port number for Squid proxy server is 3128 but we can change it any other port as per our requirement. For example, if we want to change the default port from 3128 to 8080, we can do so by making any entry for same in squid configuration file i.e. ‘/etc/squid/squid.conf‘

Open squid.conf,

$ sudo vim /etc/squid/squid.conf

& search for ‘http_port 3128’ & change it to

http_port 8080

Save file & exit. Than restart the squid service to implement the changes.

$ sudo systemctl restart squid

We can also use

$ sudo squid -k reconfigure

to implement changes, without restarting the squid service.

Limit download size of the files

We can also impose a download limit based on file size for squid proxy users. We need to provide ‘reply_max_body_size’ directive in squid.conf file to impose a file size limit.

Open squid.conf

$ sudo vim /etc/squid/squid.conf

& make the following entry in the bottom of the file

reply_body_max_size 5120 KB all

This will impose a limit of max 5Mb download for a single file for all the squid users. We can also impose limit based on user,

reply_body_max_size 5120 KB user1 user2

or based on group,

reply_body_max_size 5120 KB user1 group2

After making changes to file save it & restart or reconfigure squid to implement the changes.

Limit upload size of the files

To impose limit of size of file that can be uploaded, we use “request_body_max_size”. Make the following entry in squid.conf to impose limit on file uploads,

$ sudo vim /etc/squid/squid.conf

request_body_max_size 100kb

Save file & restart/reconfigure the squid server to implement changes. We can also impose upload limits based on users & groups, same as we imposed limits on file download.

Allow Squid proxy to cache all requests

To cache all the requests that are made in squid server, add the following directive in squid.conf file,

$ vim /etc/squid/squid.conf

cache allow all

Save file & reconfigure the squid server to implement the changes.

Disable caching of some websites

To disable cache only on some website, we will first create an ACL with websites & than disable the caching for that ACL. Make the following entries on squid.conf file,

$ vim /etc/squid/squid.conf

acl NOcache_websites dstdomain www.linuxtechlab.com www.msn.com

no_cache deny Nocache_websites

Save file & restart/reconfigure the squid server to implement changes.

Deny cache requests for some file extensions

To disable caching of files with certain extension, add the following lines in squid.conf file.

$ sudo vim /etc/squid/squid.conf

hierarchy_stoplsit .mp3 ?

hierarchy_stoplist .xls ?

acl ext1 urlpath_regex \.mp3 \?

acl ext2 urlpath_regex \.xls \?

no_cache deny ext1

no_cache deny ext2

Here, we have blocked caching for MP3 & XLS files. Save file & restart/reconfigure the squid server to implement changes.

Note:- To remove existing cached file, execute following command from terminal,

$ sudo rm -rf /var/spool/squid/*

Deny all cache requests

To deny all cache requests made to squid server, add the following directive in squid.conf file

$ vim /etc/squid/squid.conf

cache deny all

Save file & reconfigure the squid server to implement the changes.