Welcome to IP Country: A New Layer of Asterisk Security

One of the problems with writing a blog like Nerd Vittles is it's more than double the work of your typical blog where a writer pontificates about something and then moves on. What makes Nerd Vittles a little different is that, with help from a number of very gifted developers, we actually create useful applications and then write about how to use them. So you get a bonus for the same low price: free! This obviously imposes some time constraints in order to get fresh material into your hot little hands every week.

This week we turn our attention to Asterisk® Security again and unfortunately the Whole Enchilada is not yet ready. So today you get Chapter I of this topic with a comment that we're still mulling over some enhancements. When those pieces are finished or at least properly evaluated, we'll produce a sequel. Software houses spend years developing applications. And sometimes it takes us more than a week. 🙂

Let's start with a few observations which should be quite obvious to those who have wrestled with VoIP or Asterisk for a while. Internet security is a bitch. And Asterisk security is much, much worse. When a few disgruntled people can bring Twitter to its knees because they're mad about some particular tweet or Twitter user, it tells you what we're all up against. Hate to say it but we can all thank Microsoft for years of security neglect that rendered the Windows operating system less than optimum in preventing the spread and deployment of BOTs. And the tools have gotten more dangerous as well. Strangers (our euphemism for these folks) write new software, too.

If you're using PBX in a Flash (and you really should be!), you know that we've devoted enormous resources to Asterisk security. Two years ago when PBX in a Flash was introduced, the majority of people using Asterisk still were using 1234 as the extension password on all or most of their extensions. A couple $100,000 phone bills and lots of public education, and that situation hopefully is behind us. Two years ago, no Asterisk aggregation included a firewall... except PBX in a Flash. Believe it or not, there were individuals running Asterisk servers on the public Internet with a default root password of password. That added more than a few more BOTs to the Internet kettle of fish. Then there were the brute force password hacks that hit Asterisk servers thousands of times per minute guessing passwords. Nothing stood in the way of these attacks until PBX in a Flash introduced Fail2Ban which automatically blacklisted IP addresses after a certain number of failed login attempts. We followed Fail2Ban with our Atomic Flash product which provided a turnkey Hamachi VPN implementation for rock-solid safe remote computing. And, of course, there was a one-minute Hamachi VPN install script for standard PBX in a Flash systems. No other aggregation has it to this day.

The purpose of the history lesson isn't to crow about PBX in a Flash although we're mighty proud of it. Rather we wanted to make you aware that precious little development effort is actually going into security while enormous resources are devoted to things such as Internet faxing, Skype, and Google Voice integration. We'll be the first to admit that we love the latest gee whiz gizmos as much as anybody. But come on. A handful of us who do this purely for fun somehow manage to turn out loads of security enhancements while huge, for-profit companies are devoting virtually zero resources to making Asterisk, SIP, and the VoIP community safer. SIP is about as secure as whispering at a movie theater. Google releases Google Voice with SIP access protected by a 4-digit password. 🙄 That approach to security needs to change, or we're all going to wake up sorry one day soon. If this is preaching to the choir, then feel free to pass this article on to one of your brethren who has not yet seen the light! Start by reading our Primer on Asterisk Security.

If you have extremely secure passwords on your Asterisk extensions and trunks, and you have deployed a properly configured firewall with Fail2Ban to protect against brute force attacks, then you're ahead of the curve insofar as Asterisk security is concerned. But what we think is still missing is access restrictions based upon what the military calls a "need to know." Simply stated, it means folks shouldn't get access of any kind to your Asterisk server unless they have a need to be there. And, if we find someone there that doesn't belong, they should be kicked off and banned from further access.

So today we have a new security tool for your Asterisk toolbox: IP Country, country-based network filtering by IP address. In a nutshell, it means configuring your Asterisk server to dramatically reduce the number of IP addresses which can reach your server at all. If you receive anonymous SIP connections from all around the globe that you actually need or if you're attacked from a BOT running on grandma's Windows machine down the block, this may not work for you, but it's another tool in your quiver of arrows. For most servers, it has the potential to reduce the vulnerability from random outside threats substantially. It's taken a lot of research to come up with much of what follows, and we want to express our special thanks to Sandro Gauci and Joe Roper for their assistance. Some of this technology has been around for many years, but unfortunately it was expensive. So we also want to express our special appreciation to MaxMind for releasing their open source GeoLite Country database which is now free for downloading. That is the critical ingredient in much of what follows. So here's a word from our sponsor:

This product includes GeoLite data created by MaxMind, available from http://www.maxmind.com/.

Scope of Protection. An obvious question is just exactly what are we trying to protect. In our view, it's several things. First, we don't want strangers logging in to extensions on our server and making free calls around the globe using pilfered or hacked passwords. We also don't want strangers using our extensions to masquerade as us for any other purpose. Second, we don't want strangers randomly calling our server using SIP URI's that they've dreamed up. And third, we don't want strangers accessing any other applications on our server including SSH and FTP as well as web and email services.

IP Country Design. As with other security features in Asterisk, FreePBX, and IPtables, our implementation of IP Country uses permit and deny access tables that consist of authorized and unauthorized ranges of IP addresses. There's also a table with the latest GeoLite Country information which is used as the data source for your permit table. When a connection to the server is made, the IP address is checked against the permit table of authorized addresses. If there's no match, we'll consider the connection a stranger. If there is a match, then we'll check the deny table to make certain this particular IP address hasn't been banned. Unless you alter all of our scripts, your system must be using the default MySQL account name of root with a password of passw0rd. As configured in PBX in a Flash, this is NOT a security risk since MySQL access is limited to your server, and your server requires root credentials to log in.

Today's Objective. To get everyone started, we're going to tackle the first two objectives today. The solutions offered should work fine on any FreePBX-based Asterisk system... even those that hide the existence of FreePBX.

For outgoing calls, we'll introduce a new script which runs periodically to examine the IP addresses attached to every SIP and IAX extension and trunk on your Asterisk server. If a stranger's IP address is identified (as explained above), we'll add an IPtables firewall rule to permanently block access to your server from this IP address. These rules are stored in /etc/sysconfig/iptables should you ever need to remove an IP address that has been blocked. You can adjust the script execution frequency based upon the thickness of your wallet. After all, it's your phone bill. This functionality is mutually independent from the incoming call protection outlined below so you can use either or both of the functions to meet your own requirements. For systems that use enormous numbers of SIP URI's for communications around the globe, you might choose to implement just this piece for extension and trunk IP Country protection without altering your incoming dialplan at all. Keep in mind that FreePBX now supports permit and deny IP address filters on extensions, something you really should be using even if you decide against implementing the IP Country security protection layer.

For incoming calls, we're going to modify FreePBX's existing Blacklist functionality to also look up the calling IP address in our IP Country permit and deny tables. If the IP address is authorized, the call will go through. Otherwise, the call will be treated just as if the caller's number were blacklisted. Be aware that incoming calls to one of your commercial DIDs may reflect the IP address of your provider since the caller may be calling from a Plain Old Telephone rather than an IP address. The existing Blacklist functionality can be used to block these unwanted callers. If you live in the United States, you'll probably also want to call 888-382-1222 and place your DIDs in the Do Not Call database. Just call from a phone using the CallerID of the number you wish to block.

Installing GeoLite Country. To get started, log into your server as root and issue the following commands:

cd /
wget http://bestof.nerdvittles.com/applications/ipcountry/ipcountry.tgz
tar zxvf ipcountry.tgz
rm ipcountry.tgz
cd /root/ipcountry
./nv-ipcountry

Once the nv-ipcountry script begins to run, it will download and install the GeoLite Country database into MySQL. You then will be asked whether to add countries to your permit table. Since your permit table is empty at this point, the answer should be yes. You'll then get a list of country codes. Choose the two-character country code desired and type it in UPPERCASE, e.g. US. If you want to add one or more additional countries, just rerun ./nv-ipcountry and do NOT initialize the permit table (which erases all of its contents).

New GeoLite Country databases are released every month or two so get used to the procedure. You'll be using it periodically to keep your list of IP addresses current. We'll cover the update procedure after we get you up and running.

Remember: If no IP addresses for any country are added to the permit table, you will not be able to make calls or register trunks with your providers! The only default entries added to the permit table are the non-routable, private IP address ranges, e.g. 192.168.0, etc. The geolite table is merely a data repository of the latest GeoLite Country database and has no effect on the daily operation of your system! You use it only as a data source for populating your permit table.

Testing IP Country. Before we actually turn anything on, we need to be sure we're not going to blow your Asterisk system out of the water! In short, we want to make sure that every extension that's supposed to be able to make a connection to your PBX still can. And we need to make sure all of your trunk registrations still are working. While you're still in the /root/ipcountry directory, issue the following command: ./test.sh. This script will display all of your SIP and IAX connections and then will tell you whether each connection will pass muster with IP Country security in place. Each IP address should display ok. If any of them show ko, you have a problem. This means that you have an extension or trunk with an IP address that is not included in your permit table. You can scan through the show peers listings in the display to figure out which providers or extensions are associated with any problem IP addresses. Be sure it's not a bad guy first. Then you have a couple of options. You can either manually add the IP address to the permit table as outlined below. Or you can add additional countries which include the missing IP address(es). To decipher the country of any problem IP address, go to this link and plug in the IP address. Once you've made entries in your permit table to cover all of your needed IP addresses, run the test script again just to be sure everything shows ok. Do NOT proceed until you get all ok's, and don't write us if you do.

Manually Adding IP Addresses to IP Country. We've provided a command-line utility which makes it easy to add IP addresses and address ranges to either the permit or deny tables of IP Country. Be very careful using this tool! There's limited error-checking which means it's easy to create a mess. You'll find iputility.php in the /root/ipcountry folder. Since all IP addresses are stored as integers, you can use it to merely discover the integer value of an IP address, or you can actually insert IP addresses into either the permit or deny tables. Here are a few examples to show how the utility works:

A couple of points worth noting. First, all custom entries in your permit and deny tables using iputility will show a country code of AA. This makes them easy to find using phpMyAdmin if you make a mistake. Second, if you attempt to enter the same IP address range more than once, you'll get a database error since all entries in the tables must be unique. Third, remember that entries in the deny table take precedence over entries in the permit table. So, if the same IP address or address range is in both tables, access will be denied. The reason for this is to make it easy to exclude a few bad apples from a country that you might otherwise find unobjectionable. Finally, keep in mind that manual entries added to the permit table will have to be added again each time you initialize the table and insert new country IP codes after a GeoLite Country refresh. The deny table is unaffected by database refreshes. So make yourself a list of entries you manually insert into the permit table and keep it in a safe place for future reference.

Activating the IP Address Checker. In the /root/ipcountry directory, you'll find the script that we'll use to check your system periodically to be sure all of the extensions and trunks are registered at permitted IP addresses. To run the script manually, log into your server as root and type: /root/ipcountry/ip-checker.sh. When you run it, you shouldn't see any modifications to IPtables, just a string of ok's. So now we want to added the script as a cron job that will be run periodically to watch your system. Edit /etc/crontab and insert the following line at the bottom of the file:

*/1 means run the script once a minute, all day and night, every day. */5 means every 5 minutes. You make the call on how safe you'd like your system to be. If you'd like to receive an email or text message every time an IP address is blocked by ip-checker.sh, just edit the filecheck.php script, uncomment the two lines that begin with // and replace yourname@gmail.com with your email or text message address.

WARNING: For ip-checker.sh to work properly with IPtables, there are a couple of prerequisites. First, IPtables must be running on your system with the iptables file located in /etc/sysconfig. Second, your IPtables setup must include an SSH permit rule that looks like this:

-A INPUT -p tcp -m tcp --dport ssh -j ACCEPT

We use this rule as a place finder to determine where to insert new rules to block stranger's IP addresses. If you don't have the above rule, filecheck.php (used by ip-checker.sh) won't be able to insert new rules. So you'll need to manually edit filecheck.php to provide a "hook" that can be used to insert rules into your iptables file. PBX in a Flash systems come preconfigured to support this. With other aggregations, YMMV!

Activating the Incoming Call Checker. To screen incoming calls using your IP Country permit and deny tables, the setup is straight-forward assuming you are running the latest version of FreePBX 2.5. We're going to adjust the Blacklist context to also perform IP address lookups from IP Country when new calls arrive on your PBX. Just log into your server as root and add the following lines to the bottom of the extensions_override_freepbx.conf file in /etc/asterisk:

Make sure you remove the line-wrap in the s,n(hasat) line and any others that may have wrapped in the display above! Then save the file and reload your Asterisk dialplan: asterisk -rx "dialplan reload". You're all set! If you'd like email notices when a stranger calls and is blacklisted, edit nv-ipcountry.php in /var/lib/asterisk/agi-bin. Plug in your actual email address in the $email variable and set $emailalerts = 1.

Housekeeping 101. As we mentioned above, the pool and location of IP addresses continues to change so periodic updates are necessary, or you'll end up blocking calls that otherwise should be permitted. MaxMind updates GeoLite Country on the first day of every month so add it to your TO-DO list. We strongly recommend that you perform these steps through an SSH connection from a remote PC. Why? Because, if you forget step 1 while logged directly into your server, you could inadvertently lock yourself out of your own system if the ip-checker script happens to run while your permit table is empty. If you do it from a remote machine, you can simply move to another machine and follow these instructions properly. Otherwise, you've got a serious problem on your main server. If this server provides phones to your business, do the update when the server is idle. So here's the drill:

1. Comment out the ip-checker.sh /etc/crontab entry
2. Download new GeoLite Country database from MaxMind
3. Initialize the ipcountry.permit table
4. Add authorized countries back into ipcountry.permit table
5. Add back any custom entries to permit table
6. Test your IP Country system to make sure you get all ok's
7. Reactivate ip-checker.sh in /etc/crontab

1. Log into your server as root. To comment out the ip-checker.sh line in /etc/crontab, just add # as the first character on the line and save the file.

2. Change to the /root/ipcountry directory and run ./nv-GeoIPrefresh.

3. While still in the /root/ipcountry directory, run ./nv-ipcountry and choose 1-Yes to initialize your ipcountry.permit table.

4. Continue running or rerun ./nv-ipcountry to add each desired country to your ipcountry.permit table.

5. Run ./iputility.php to add custom IP address entries to your ipcountry.permit table. You do NOT need to reenter addresses in the deny table. It is unaffected by this update procedure.

6. Test your system again to make sure all extensions and trunks get an ok by running ./test.sh.

7. Edit /etc/crontab and remove the # at the beginning of the ip-checker.sh line and save the file.

What's Next. We're still exploring another possibility with IP Country, and that is integrating GeoLite Country directly into IPtables. This would validate every packet coming into your firewall using IP Country-like rules in IPtables. If you want to look at how it could be done, see this excellent writeup. Well, not so fast. Unfortunately, it won't compile under CentOS 5.2. Here's a link to the problem code if there are any Linux gurus in the house. Our reluctance in doing this has to do with performance. Keep in mind that, without stateful packet inspection, every single packet coming into your server would presumably trigger a database lookup. On a busy telephony system generating hundreds of thousands of packets per second, it would take a beast of a server with sufficient memory to cache the entire IP Country database in order to handle the processing load. So now we've got to either learn about or find an expert on the IPtables State Machine. If anyone wants to experiment, please share your expertise with the rest of us. There's a Google Voice invite in it for you, too.

whos.amung.us If you're wondering what your fellow man is reading on Nerd Vittles these days, wonder no more. Visit our new whos.amung.us statistical web site and check out what's happening. It's a terrific resource both for us and for you.

Need help with Asterisk? Visit the PBX in a Flash Forum.
Or Try the New, Free PBX in a Flash Conference Bridge.

Special Thanks to Our Generous Sponsors

FULL DISCLOSURE: ClearlyIP, Skyetel, Vitelity, DigitalOcean, Vultr, VoIP.ms, 3CX, Sangoma, TelecomsXchange and VitalPBX have provided financial support to Nerd Vittles and our open source projects through advertising, referral revenue, and/or merchandise. As an Amazon Associate and Best Buy Affiliate, we also earn from qualifying purchases. We’ve chosen these providers not the other way around. Our decisions are based upon their corporate reputation and the quality of their offerings and pricing. Our recommendations regarding technology are reached without regard to financial compensation except in situations in which comparable products at comparable pricing are available from multiple sources. In this limited case, we support our sponsors because our sponsors support us.

BOGO Bonaza: Enjoy state-of-the-art VoIP service with a $10 credit and half-price SIP service on up to $500 of Skyetel trunking with free number porting when you fund your Skyetel account. No limits on number of simultaneous calls. Quadruple data center redundancy. $25 monthly minimum spend required. Tutorial and sign up details are here.

The lynchpin of Incredible PBX 2020 and beyond is ClearlyIP components which bring management of FreePBX modules and SIP phone integration to a level never before available with any other Asterisk distribution. And now you can configure and reconfigure your new Incredible PBX phones from the convenience of the Incredible PBX GUI.

VitalPBX is perhaps the fastest-growing PBX offering based upon Asterisk with an installed presence in more than 100 countries worldwide. VitalPBX has generously provided a customized White Label version of Incredible PBX tailored for use with all Incredible PBX and VitalPBX custom applications. Follow this link for a free test drive!
 

Special Thanks to Vitelity. Vitelity is now Voyant Communications and has halted new registrations for the time being. Our special thanks to Vitelity for their unwavering financial support over many years and to the many Nerd Vittles readers who continue to enjoy the benefits of their service offerings. We will keep everyone posted on further developments.
 

Some Recent Nerd Vittles Articles of Interest...