Avoiding the $100,000 Phone Bill: A Primer on Asterisk Security

Here's a headline to wake up any CEO: "Small business gets $120,000 phone bill after hackers attack VoIP phone." News.com.au actually ran this story on January 20. "Criminals hacked into an Internet phone system and used it to make 11,000 international calls in just 46 hours... 115,000 international mobile calls were made using the small business's VoIP system over a six month period."

News Flash: Be sure to read our latest article introducing Travelin' Man 3, a completely new security methodology based upon FQDN Whitelists and DDNS. In a nutshell, you get set-it-and-forget-it convenience and rock-solid VoIP security for your Cloud-based PBX or any PBX in a Flash server that's lacking a hardware-based firewall and you get both transparent connectivity and security for your mobile or remote workforce.

For the latest Security Tips: See our most recent article.

Sad to say that folks install VoIP phone systems to save money and then completely ignore tried-and-true network security principles: hardening your system, regularly watching your logs, and periodically changing your passwords. If PBX in a Flash were a commercial offering, we'd probably keep much of what follows to ourselves and start touting our PBX systems as the only Asterisk® offering with Secure-Wrap™. That's not our world, of course, nor is it what open source is all about... which turns out to be both a blessing and a curse. We openly and jointly figure out ways to secure our Asterisk systems as well as those of our competitors. Then the bad guys get to read all about it and come up with new, more creative "solutions." The silver lining is there are millions of insecure Asterisk systems so the creeps typically move on to easier targets.

Today we'll walk you through our Top Ten Security Tips and Tricks. All of these can be implemented easily to harden your Asterisk PBX and lessen the chances of the bad guys transforming your VoIP system into a free, international payphone: you pay, they phone. In the process, we'll identify some common security blunders that accompany new system installs in hopes that you won't make the same mistakes. So let's start with the basics. If you plug your Asterisk PBX directly into the public Internet without carefully securing it, your chances of being hacked within the hour are pretty good.

Rule #1: Protect Your PBX With IPtables. PBX in a Flash systems are delivered with the IPtables firewall enabled. Leave it that way! If your Asterisk implementation doesn't have IPtables support, demand that it be added immediately or ask for assistance in adding it yourself. There is no reason not to use a freely available, open source firewall, period! And there are many good tools including WebMin (also included in PBX in a Flash distributions) to get it configured properly. With PBX in a Flash, all of the grunt work has been done for you.

Firewalls, of course, are only as good as the set of rules defined to secure your system. So only activate ports that are absolutely essential to run your PBX. For an excellent review of the ports that are opened by default in PBX in a Flash systems, see Joe Roper's summary. Think of an activated port as a hole in the dike. The more holes you add, the less secure your PBX will be. We'll leave it to you to count the holes in the dike if you choose to run your PBX without IPtables enabled. Our rule of thumb for PBX security goes something like this. If you don't need web access to your PBX, don't open ports 80 and 9080. If you don't need SSH, FTP, FOP, or WebMin access to your PBX, don't enable those ports. Better yet, don't even turn those services on unless there is a pressing need.

All of the IPtables rules are stored in /etc/sysconfig/iptables. Don't edit this file unless you know what you're doing. If you need help with the rules, post a question on the PBX in a Flash Forum. Typical response time on posted questions is under an hour on our forum. And don't forget to restart IPtables if you make changes to any of the rules: service iptables restart.

Rule #2: Protect Your PBX With A Hardware-Based Firewall. If one firewall is good protection, two firewalls are even better. As much as NAT-based firewall/routers get a bad rap, the extra layer of protection that a $50 hardware-based firewall/router delivers cannot be overstressed. Think of the software-based firewall as the tool of choice to secure your PBX on your internal LAN while the hardware-based firewall secures your system on the public Internet. We recommend the dLink WBR-2310 for home and SOHO use. It provides a reliable NAT-based router, a firewall, and excellent WiFi capability for under $50. If you've got some spare change, step up to one of dLink's Gaming Routers which we happen to use. They provide all the tools you'll need to prioritize your VoIP traffic. As with Rule #1, only open and redirect ports that are absolutely essential to use your PBX.

Rule #3: Safeguard Against Random Password Hacks. There is no better tool to protect your PBX from random password attacks than Fail2Ban 0.8.3. Fail2ban scans log files and bans IP addresses that make repeated, unsuccessful password attempts. It updates IPtables rules to reject those IP addresses for a period of time that you can set in /etc/fail2ban/jail.conf. Originally PBX in a Flash systems were shipped with an earlier version of Fail2Ban that provided only minimal protection. If your system doesn't include the jail.conf file above, you still have the older version. Simply run our update script to get the current release:

cd /root
mkdir fail2ban
cd fail2ban
wget http://pbxinaflash.net/source/fail2ban/fail2ban-update
chmod +x fail2ban-update
./fail2ban-update
service fail2ban restart

As was true with IPtables, Fail2Ban is only as good as the rules which are defined to identify failed password attempts on your system. On PBX in a Flash systems, we now protect against web, FTP, SSH, SIP, and IAX password attempts.

If your particular Asterisk implementation lacks Fail2Ban support, you're missing a critically important (free) tool to safeguard your system from random password attacks against SSH and your protected web sites as well as your SIP and IAX extension passwords. For tips on installation, review our script that is available on this thread in the PBX in a Flash Forum.

Rule #4: Narrow Access With IP Address Restrictions. Security privileges in the U.S. government are based upon a "need to know." It's pretty simple. If you don't have a need to know the information to perform your duties, you don't get the privilege. You can use a similar technique to secure your PBX by implementing IP address restrictions. For example, if all of your extensions are housed on a private subnet of your internal LAN, then there is no reason to allow Internet access to those extensions. Similarly, for extensions outside your local network, you now can hardcode the IP address into the extension to restrict access. To implement this with Asterisk and FreePBX-based systems, you'll first need to upgrade FreePBX to at least version 2.5.1.1. Once you've upgraded, go into each extension and enter either an IP address or an IP subnet for that extension in the permit field. For an IP address, the syntax is 192.168.0.44/255.255.255.255. For an IP subnet, the syntax would look like this: 192.168.0.0/255.255.255.0. This one tip would have been worth $120,000 to the Australian company referenced above. Yes, consultants can be worth their weight in gold. :-)

If you're as absent-minded as we are, you don't want to have to worry about remembering this each time you add a new extension to your system. So it's quite simple to change the default permit entry from 0.0.0.0/0.0.0.0 to the subnet mask of your LAN. Then you only have to adjust this entry whenever you add an extension which is not on your internal LAN. For example, if your LAN subnet is 192.168.0, then we want to replace the default entry with 192.168.0.0/255.255.255.0. The file to edit is /var/www/html/admin/modules/core/functions.inc.php. Just search for $tmparr['permit'] in BOTH the iax2 and sip sections of the file and make the value substitution preserving the single quotes on both sides of your new entries.

You also can implement both password and IP address restrictions to limit web access to your server. With Apache web servers, this is done through .htaccess files and directory restrictions in your Apache config files. On PBX in a Flash systems, htaccess password restrictions now are the default setup in all of our builds. Suffice it to say, if you can access the /admin directory on your web site from the Internet without being prompted for a password, your site probably has been compromised. Keep in mind that these passwords get cached so be sure you have cleaned out your browser cache before having a heart attack. Better yet, try this from a browser you don't ordinarily use (such as the one on your cellphone).

For additional security, you can further restrict access to your web directories by adding a list of authorized IP addresses to the .htaccess file in each subdirectory. Here's what an .htaccess file with IP address restrictions might look like. The first Allow entry is the private LAN subnet, the second is a remote site, and the third is the Hamachi VPN subnet mask:

Deny from All
Allow from 192.168.0
Allow from 68.218.222.70
Allow from 5.67

Rule #5: Don't Use 'Normal Ports' for Internet Access. Think of network and PBX security as a shell game. You want to do as many things differently as possible to make it as difficult as possible for the bad guys to figure out what you've done. Read that last sentence again. It's important! With a hardware-based firewall such as the WBR-2310, this is incredibly easy. dLink calls them Virtual Servers. Here is a typical entry:

HTTP   192.168.0.150   TCP 80/2319   Allow All   Always

You can simply redirect common ports to different ports for Internet access. Don't do this for SIP and IAX ports, but it works great for HTTP, FTP, and SSH access. For example, port 80 typically is the default web server port on Asterisk aggregations, and this port normally can be used on your internal LAN assuming you know and trust your users. For external (aka Internet) web access, simply remap TCP port 80 to some obscure port and change it periodically. For example, you might redirect TCP port 80 to port 2319. Once the setting is saved, you access the web site with a browser entry like this: http://pbx.mydomain.com:2319/. Then (and just as important!) next month, change the port to 4382, then 6109, and so on. Don't use these numbers obviously! Make up your own. The key here is that 5 minutes work every month will keep web access to your PBX much more secure than letting every Tom, Dick, and Ivan hammer away at port 80 every night while you're sleeping. Incidentally, most of these routers also will let you block access to certain ports during certain hours of the day. If you're sleeping, there's really not much need to provide SSH and web access to your Asterisk server. At the risk of being labeled xenophobic, keep in mind that many of the world's best crackers reside in countries where daytime happens to be nighttime in the United States.

Rule #6: Really Secure Passwords Really Do Matter. While we have no hard evidence to back this up, our wild-assed guess (WAG) is that 90% of the security breaches in Asterisk systems have been the direct result of folks using passwords that matched the extension numbers on their phone systems. Since most Asterisk PBX systems are configured with extension numbers beginning in the 200, 700, or 800 range of numbers, it really wasn't Rocket Science to remotely log into these servers and make unlimited SIP telephone calls. The first five rules would have protected most Asterisk systems. But our WAG on the number of Asterisk PBX's that have implemented all five rules above would be less than one in a thousand. Part of that is because some of these tools weren't readily available until recently. But part of it is because most of us are just plain L-A-Z-Y.

Really secure passwords really do matter. And it's more than having a secure root password. All of your passwords need to be secure including those on your phone extensions and voicemail accounts unless you are absolutely certain that you have blocked all access to your system from everyone except trusted users. If you use DISA, make certain it has a really, really secure password. Part of having really secure passwords is regularly changing them. And our rule of thumb on Asterisk system passwords goes one step further. Never, ever use passwords on your PBX that you use for other important personal information (such as financial accounts). You've been warned. It's your phone bill and bank account!
<end of sermon>

Rule #7: Minimize Web Access To Your PBX. Most of the Asterisk aggregations utilize FreePBX as the graphical user interface to configure your Asterisk PBX. Because FreePBX is web-based, it is extremely dangerous to leave it exposed on the Internet. As much as we love FreePBX, keep in mind that it was written by dozens and dozens of contributors of various skill levels over a very long period of time. Spaghetti code doesn't begin to describe some of what lies under the FreePBX covers. Make absolutely certain that you have .htaccess password protection in place for all web directories in at least these directory trees: admin, maint, meetme, and panel.

Our rule of thumb on Internet web accessibility to an Asterisk PBX goes like this. Don't! But, if you must, build as many layers of protection as possible to assure that your system is not compromised. If the bad guys get into FreePBX, the security of your PBX has been compromised... permanently! This means you need to start over with all-new passwords by installing a fresh system. You simply cannot fix every possible hole that has been opened on a FreePBX-compromised system!

Rule #8: Implement VPNs for PBX Systems. PBX in a Flash has provided simple install scripts to deploy Hamachi VPNs on all of our current systems. Hopefully, the other aggregations will do likewise. In addition, we offer turnkey VPN in a Flash systems which provide this functionality out of the box. VPNs provide an incredibly simple way to interconnect PBX systems worldwide and assure secure communications between these interconnected systems. We now are exploring other VPN solutions which would facilitate the use of VPN-enabled telephones such as the new offerings from SNOM.

Rule #9: Check Your Logs Every Day. We're still dumbfounded by the following quote from the article above: "115,000 international mobile calls were made using the small business's VoIP system over a six month period." Six months and they never checked their call logs? Sounds like they earned this phone bill. FreePBX provides an incredibly simple way to review your call logs. Click the Reports tab at the top of the screen and look at the bar graph showing the number of calls each day and the combined length of those calls. Nothing could be easier. Do it every single day! It also should be noted that Ethan Schroeder has released a beta of some new monitoring software which will provide more granular monitoring of daily call volumes. For additional information or to participate in the beta, visit this link.

Rule #10: Do Some Reading... Regularly. No security implementation is complete without a little regular effort on your part: reading. If you're going to manage your own network or PBX, then you need to keep abreast of what's happening in the business. There are any number of ways to do this, none of which take much time. The simplest approach is just to scan the Open Discussion, Add-Ons, and Bug Reporting topics on the PBX in a Flash Forum, the trixbox Forum, and the FreePBX Forum. Aside from reviewing your call logs, it's the best 15 minutes you could spend to safeguard your system. We also have an RSS Feed which includes security alerts.

Update #1: Be sure to read this great new article. It has two fresh ideas for securing your system!

Update #2: Please also read this Nerd Vittles Alert about FreePBX backdoors and default passwords that was published on April 15, 2011.

Some Other Suggestions. A couple other suggestions come to mind that don't involve securing your PBX per se but nevertheless will lessen your exposure in the event of a security breach. First, if your usual calling patterns don't involve international calling or if they're limited to one or two countries, tighten up your outbound dialplan and restrict calling to countries that you actually need. It can always be changed when the need to call elsewhere arises. Second, if you use pay-as-you-go providers, never use credit card auto-replenishment. Instead, add funds periodically using the provider's web interface. The advantage of this is that, if someone does manage to break into your system, your loss will be limited to the current balance in your provider account. You'll not only save a lot of money, but you'll also get a notification that something has gone horribly wrong. Finally, a forum user mentioned one we had overlooked. If you have a mix of POTS and VoIP lines, don't put the POTS lines in the default outbound pool for toll calls. This could potentially save you lots of money.

Continue Reading Part II: The VoIP WhiteList for IPtables...

Got Some Other Ideas? 50,000 heads always are better than one when it comes to network security. If there are things we've missed, take a minute to post a comment. It'll help all of us keep our systems more secure. Good luck!

Digium® Weighs In. Since this article first appeared, Digium has released its own set of tips on SIP security. By all means, have a look!


Security Alert of the Week. A trixbox user yesterday reported that he had discovered a rootkit exploit on his server. You can could read all about it here. The 6:03 a.m. (California time) post mysteriously disappeared a few hours later... soon after the trixbox staff got to work. Another darn computer failure according to Fonality staff. :-? We've attempted to recreate the information from Google snippets. And here's a simple test to see if you have a similar rootkit problem:

ls -all /sbin/init.zk


Want a Bootable PBX in a Flash Drive? Our bootable USB flash installer for PBX in a Flash will provide all of the goodies in the VPN in a Flash system featured last month on Nerd Vittles. You can build a complete turnkey system using almost any current generation PC with a SATA drive and our flash installer in less than 15 minutes!

If you'd like to put your name in the hat for a chance to win a free one delivered to your door, just post a comment with your best PBX in a Flash story.1

Be sure to include your real email address which will not be posted. The winner will be chosen by drawing an email address out of a hat (the old fashioned way!) from all of the comments posted over the next couple weeks. All of the individuals whose comments were used in today's story will automatically be included in the drawing as well. Good luck to everyone and Happy New Year!!


New Fonica Special. If you want to communicate with the rest of the telephones in the world, then you'll need a way to route outbound calls (terminations) to their destination. For outbound calling, we recommend you establish accounts with several providers. We've included two of the very best! These include Joe Roper's new service for PBX in a Flash as well as our old favorite, Vitelity. To get started with the Fonica service, just visit the web site and register. You can choose penny a minute service in the U.S. Or premium service is available for a bit more. Try both. You've got nothing to lose! In addition, Fonica offers some of the best international calling rates in the world. And Joe Roper has almost a decade of experience configuring and managing these services. So we have little doubt that you'll love the service AND the support. To sign up in the USA and be charged in U.S. Dollars, sign up here. To sign up for the European Service and be charged in Euros, sign up here. See the Fonica image which tells you everything you need to know about this terrific new offering. In addition to being first rate service, Fonica is one of the least expensive and most reliable providers on the planet.
 
New Vitelity Special. Vitelity has generously offered a new discount for PBX in a Flash users. You now can get an almost half-price DID and 60 free minutes from our special Vitelity sign-up link. If you're seeking the best flexibility in choosing an area code and phone number plus the lowest entry level pricing plus high quality calls, then Vitelity is the hands-down winner. Vitelity provides Tier A DID inbound service in over 3,000 rate centers throughout the US and Canada. And, when you use our special link to sign up, the Nerd Vittles and PBX in a Flash projects get a few shekels down the road while you get an incredible signup deal as well. The going rate for Vitelity's DID service is $7.95 a month which includes up to 4,000 incoming minutes on two simultaneous channels with terminations priced at 1.45¢ per minute. Not any more! For PBX in a Flash users, here's a deal you can't (and shouldn't) refuse! Sign up now, and you can purchase a Tier A DID with unlimited incoming calls for just $3.99 a month and you get a free hour of outbound calling to test out their call quality. To check availability of local numbers and tiers of service from Vitelity, click here. Do not use this link to order your DIDs, or you won't get the special pricing! After the free hour of outbound calling, Vitelity's rate is just 1.44¢ per minute for outbound calls in the U.S. There is a $35 prepay when you sign up. This covers future usage and any balance is fully refundable if you decide to discontinue service with Vitelity.
 


Some Recent Nerd Vittles Articles of Interest...

Be Sociable, Share!

  1. This offer does not extend to those in jurisdictions in which our offer or your participation may be regulated or prohibited by statute or regulation. []

18 Responses to “Avoiding the $100,000 Phone Bill: A Primer on Asterisk Security”

  1. One more tip… Update your asterisk regularly…
    I had a former client who would not update because he did not want to pay us the $50 we charged to do the security update that was posted after the many asterisk hacked on November, he got a bill for $500 over the usual bill, he called us and bought the maintenance contract.

    If you install for other keep a record of your clients and every time there is a security update call your clients, explain the issue and try to make them a deal to update! It give us all a bad name if asterisk gets hacked. Please be responsible.

    BTW Great post Ward, security is often overlooked, and is one of the most important parts of any network, let alone a PBX.

  2. Reginald W says:

    Thanks for the heads up and training on securing a PIAF system. It is always useful to know these things.

    I remember reading about turning OFF Plug N Play in routers as there is ZERO security on it from the users network, so that if a computer was compromised INSIDE the network, it could access the router and open up holes in the router. I have it turned OFF and have told others about turning it OFF as well. Unfortunately it is ON by default on my router. It would be nice if this was OFF by default on all routers, or if the router could be updated with it being OFF instead of being on by default. Hope that helps fill another hole in the wall.

  3. Reginald W says:

    One other thing that I’ve been thinking of is what if you had TWO hardware routers on your network. The first, connected to your main internet pipe would have wifi access to allow any computer to get wireless if needed. The second would be wired ONLY and which would have the PIAF system sitting behind it, thus requiring anyone trying to get in to have to go through two layers of security to get access. Don’t know how well this would work. I thought of doing this for an internal network or even two internal networks to cut down on the amount of traffic on each leg of the network. I don’t know enough about how the router/switches would handle the traffic out to each leg, how different router/switches would interact with each other and whether traffic would bottleneck and make it unworkable. I haven’t had time to set up a wired-only router to test it to see if this is something that would work well.

    It should/would cut down on the amount of logs to sort through, as anything that came in through the net that tried to get into the wired only network should be more easily seen and acted upon. The main network would be like a fence around the house whie the second network would be the house itself and thus (theoretically) more secure if the proper security measures are implemented on each network.

    For the paranoid, a third network would be like the safe inside the house. Remember to poke holes in the tin foil hat that surrounds the router or it will get too hot and stop working.

  4. Ward, excellent article. When XP came out, it was fun for a while to connect a fresh XP box directly to the internet and use a stopwatch to time how long before it became compromised. Usually under a minute.

    The same can be said of almost anything you place on the net these days.

    I applaud the efforts the PiaF team has made to make the PiaF distro easy to lockdown, use fail2ban and Hamachi VPN.

    I have good stories, but this is to say thanks and I am happy to purchase a PiaF Flash if they are available.

    cosmicwombat

  5. John Senay says:

    Ward,

    I think you need a good tutorial on how to use ssh to build a tunnel between the outside system to the internal asterisk system behind the firewall.

    http://thinkhole.org/wp/2006/05/10/howto-secure-firefox-and-im-with-putty/

    If someone has to have external gui access, use ssh to set it up.

    JJS

  6. Andrew says:

    One additional thing I do b/c I have phones that connect to my server using dynamic IPs is that I add the PIN feature to any long distance or international calls. Each person is assigned their own PIN. It’s a small thing, but it’s one extra step to prevent automated calling in case someone did manage to guess our extensions and passwords.

  7. Dan says:

    I installed PiaF 1.1 and have updated periodically since. I don’t seem to have fail2ban installed at all. I tried to run the fail2ban-update script, but it says I need to have fail2ban installed first. What’s the recommended way to install fail2ban?

    [WM: Get your existing system up to date with the latest version of FreePBX and do a FreePBX backup. Copy that backup to another machine. Then install the latest version of PBX in a Flash and get that machine up to the latest version of FreePBX. Create a FreePBX backup. Now copy the off-site backup you previously made to the new backup directory and restore it using FreePBX. Finally, run the fail2ban-update script again, and you’ll be all set.]

  8. Ed says:

    Hi Ward,

    Your points about hardening systems are all spot-on. Sad to say I got hacked a few months ago in spite of having hardened my system (FreePBX 1.4).

    I check my logs daily, so I discovered a spike in call volume pretty quickly. I took the PBX offline to make backups of all the log files and configuration, then started my analysis to find and close the breach.

    It turns out the hacker found my open SIP ports and kept trying to authenticate with various extension/password combinations until he/she found one that worked. They were then able to log into an extenstion on my PBX remotely and place outbound calls. My passwords were short enough that they could be brute-forced and have since been changed.

    I’ll be upgrading to FreePBX 1.5, but want to know if under FreePBX 1.4 there’s any way to prevent extensions from logging in remotely.

    If I can add to your list, I’d offer the following suggestions:

    * Use the PIN feature to restrict access for international calls, or block them altogether. I also use an ENUM trunk to see if the call can be made for free.
    * Make sure your passwords are long and complex. They’re stored in your device configuration so you don’t need to enter them each time — you can afford to make one you won’t remember!

    And, of course, check your logs regularly!

  9. Ward,

    Excellent information! This is exactly what users, admins, and especially bean-counters need to know about. A small investment in security can yield big dividends, the best of which is probably being able to sleep at night. :)

    For those concerned about telecom system security you might also want to check out the way FreeSWITCH does things. It is “secure by default” as the devs are properly paranoid.

    -MC

    [WM: We like paranoid. We all sleep better. :-) ]

  10. Michael Orr says:

    Excellent article. I have a trixbox system currently powered down. I may have been hacked since the box was making connections out to someone, haven’t had a chance to do an analysis. SBC suspended my DSL supposedly because something from my home network was phishing. Your article and my recent experience just confirms I didn’t harden the system properly. Going to replace the trixbox with PBX in a Flash..

  11. Adam says:

    Another option is to use prepaid SIP where account shuts down after the funding is exhausted. I feed mine $20 at a time and that is all I have at risk.

  12. Steve Davies says:

    I just helped somebody whose box was hacked – and it prompts me just to post.

    The article is very much focussed on IP security. But this site was exploited via Asterisk voicemail. They had the “dialout” option enabled in voicemail.conf and the hackers gained access through an insecure mailbox PIN and used the dialout feature to make international calls.

    So be very careful about the “dialout” and “callback” options in voicemail.conf.

    Regards,
    Steve

  13. dimmyr says:

    Speaking out of almost complete ignorance about VOIP VPNs – if I connect to my DD-WRT router or a Windows machine with PPTP using my iPhone, and made a SIP call using iSip or any other iPhone compatible software, would the call be encrypted?

    [WM: If the call is passing through the VPN tunnel it will be encrypted.]

  14. carl says:

    Guys For a novice the default security errors on the status page show no method of fixing:
    #Default SQL Password Used
    #Warning Default Asterisk Manager Password Used

    How are these changed?

    [WM: If you are running PBX in a Flash, both of these applications are protected by the root and FreePBX passwords. You can hide the warning messages by following the instructions at this link.]

  15. bumi akin says:

    and just a few days ago, this actually happened to someone else. see the story at ComputerWorld: “Security Manager’s Journal: Slammed with a $100,000 phone bill”

  16. ward says:

    For an interesting look at our current efforts to incorporate a VoIP Blacklist into PBX in a Flash, see this thread on the forums.

  17. ward says:

    And here’s another good reminder for those that leave their WiFi routers unprotected. This could be you!

  18. Alex Inoa says:

    This is an awesome article

Ringbinder theme by Themocracy