About once a year, we try to shine the spotlight on Asterisk® security in hopes of saving lots of organizations and individuals a little bit (or a lot) of money. The problem with open source phone systems is they’re open source phone systems. So the bad guys can figure out how they work just like the good guys. That’s not to suggest that proprietary phone systems are any more secure. They’re not. It just may take the bad guys a little longer to figure out where the holes are.

Olle Johansson has been one of the primary shakers and movers when it comes to educating folks on Asterisk security and inspiring developers to do a better job designing these systems. If you didn’t attend last year’s AstriCon and haven’t watched the Security Master Class, put it on your Bucket List. It’s free and well worth your time.

When we began building out Incredible PBX™ on other platforms this summer, we decided it was an opportune time to revisit our Asterisk security model and make it as bullet-proof as possible given the number of people now deploying Asterisk servers in the cloud. As a practical matter, there are no hardware-based firewalls to protect you with many of the cloud-based systems. So you literally live or die based upon the strength of your own software-based security model.

As in the past, security is all about layers of protection. A bundle of sticks is harder to break than a single stick. In the last month, we have rolled out new Incredible PBX systems for CentOS 7, Scientific Linux 7, Ubuntu 14, and the latest Raspbian OS for the Raspberry Pi B+. We’re in the final testing stage for a new Incredible PBX for CentOS 6.5 and Scientific Linux 6.5 as well as Ubuntu 14. All of these releases include the new Incredible PBX security model, and we will retrofit it to Fedora 20 and our standard builds for PBX in a Flash and RasPBX in coming weeks. Here’s how it works…

The 7 Security Layers include the following, and we will go into the details below:

1. Preconfigured IPtables Linux Firewall
2. Preconfigured Travelin’ Man 3 WhiteLists
3. Randomized Port Knocker for Remote Access
4. TM4 WhiteListing by Telephone (optional)
5. Fail2Ban
6. Randomized Ultra-Secure Passwords
7. Automatic Security Updates & Bug Fixes

1. IPtables Linux Firewall. Yes, we’ve had IPtables in place with PBX in a Flash for many years. And, yes, it was partially locked down in previous Incredible PBX releases if you chose to deploy Travelin’ Man 3. Now it’s automatically locked down, period. As installed, the new Incredible PBX limits login access to your server to those on your private LAN (if any) and anyone logging in from the server’s public or private IP address and the public IP address of the desktop machine used to install the Incredible PBX software. If you or your users need access from other computers or phones, those addresses can be added quickly using either the Travelin’ Man 3 tools (add-ip and add-fqdn) or using the Port Knocker application running on your desktop or smartphone. All you need is your randomized 3 codes for the knock. You can also enable a remote IP address by telephone. Keep reading!

2. Travelin’ Man 3 WhiteLists. As in the past, many of the major SIP providers have been whitelisted in the default setup so that you can quickly add new service without worrying about firewall access. These are providers that we’ve used over the years. The preconfigured providers include Vitelity (outbound1.vitelity.net and inbound1.vitelity.net), Google Voice (talk.google.com), VoIP.ms (city.voip.ms), DIDforsale (209.216.2.211), CallCentric (callcentric.com), and also VoIPStreet.com (chi-out.voipstreet.com plus chi-in.voipstreet.com), Les.net (did.voip.les.net), Future-Nine, AxVoice (magnum.axvoice.com), SIP2SIP (proxy.sipthor.net), VoIPMyWay (sip.voipwelcome.com), Obivoice/Vestalink (sms.intelafone.com), Teliax, and IPkall. You are, of course, free to add other providers or users using the whitelist tools being provided. add-ip lets you add an IP address to your whitelist. add-fqdn lets you add a fully-qualified domain name to your whitelist. del-acct lets you remove an entry from your whitelist. Because FQDNs cause problems with IPtables if the FQDN happens to be invalid or non-functional, we’ve provided a customized iptables-restart tool which will filter out bad FQDNs and start up IPtables without the problematic entries.

Be advised that whitelist entries created with PortKnocker are stored in RAM, not in your IPtables file. These RAM entries will get blown out of the water whenever your system is restarted OR if IPtables is restarted. Stated another way, PortKnocker should be used as a stopgap tool to get new IP addresses qualified quickly. If these addresses need access for more than a few hours, then the Travelin’ Man 3 tools should be used to add them to your IPtables whitelist. If your whitelist setup includes dynamic IP addresses, be aware that using ipchecker in a cron job to test for changing dynamic IP addresses will remove PortKnocker whitelist RAM entries whenever an IP address change triggers an iptables-restart.

For more detail on Travelin’ Man 3, review our original tutorial.

3. PortKnocker WhiteListing. We wrote about PortKnocker several weeks ago and won’t repeat the article here. In a nutshell, it lets you knock on three ports on a host machine in the proper order to gain access. If you get the timing and sequence right, the IP address from which you knocked gets whitelisted for access to the server… with appropriate admin or root passwords, of course. The knocking can be accomplished with either a command line tool or an iOS or Android app using your smartphone or tablet. As noted above, it’s a terrific stopgap tool to let you or your users gain quick access to your server. For the reasons we’ve documented, don’t forget that it’s a stopgap tool. Don’t use it as a replacement for Travelin’ Man 3 whitelists unless you don’t plan to deploy dynamic IP address automatic updating. Just to repeat, PortKnocker whitelists get destroyed whenever IPtables is restarted or your server is rebooted. You’ve been warned.

4. TM4 WhiteListing by Telephone. Newer releases of Incredible PBX are preconfigured with ODBC support for telephony applications. One worth mentioning is our new Travelin’ Man 4 utility which lets a remote user dial into a dedicated DID and register an IP address to be whitelisted on the server. Within a couple minutes, the user will be sent an email confirming that the IP address has been whitelisted and remote access is now enabled. For phone systems and administrators supporting hundreds of remote users, this new feature will be a welcome addition. It can be configured in a couple minutes by following the Installation instructions in the Travelin’ Man 4 tutorial. Unlike PortKnocker, whitelisted IP addresses added with TM4 are permanent until modified by the remote user or deleted by the administrator.

5. Fail2Ban. We’ve never been a big fan of Fail2Ban which scans your logs and blacklists IP addresses after several failed attempts to log in or register with SSH or Apache or Asterisk. The reason is because of documented cases where attacks from powerful servers (think: Amazon) completely overpower a machine and delay execution of Fail2Ban log scanning until tens of thousands of registration attempts have been launched. The FreePBX folks are working on a methodology to move failed login attempts to a separate (smaller) log which would go a long way toward eliminating the log scanning bottleneck. In the the meantime, Fail2Ban is included, and it works when it works. But don’t count on it as your only security layer.

6. Randomized Passwords. With the new security model described above, we’ve dispensed with Apache security to protect FreePBX® access. These new Incredible PBX releases rely upon the FreePBX security model which relies upon encrypted passwords stored in MySQL or MariaDB. As part of the installation process, Incredible PBX randomizes ALL FreePBX passwords including those for the default 701 extension as well as the admin password. When your new Incredible PBX install completes, the most important things to remember are your (randomized) FreePBX admin password AND the (randomized) 3 ports required for Port Knocker access. Put them in a safe place. Sooner or later, you’ll need them. You can review your PortKnocker settings in /root/knock.FAQ. We’ve also included admin-pw-change in the /root folder for those that are too lazy to heed our advice. With the new security model, there is no way to look up your admin password. All you can do is change it… assuming you haven’t also forgotten your root password.

7. Automatic Update Service. All new Incredible PBX builds include an automatic update service to provide security patches and bug fixes whenever you log into your server as root. If you don’t want the updates for some reason, you can delete the /root/update* file from your server. If the cost of maintaining this service becomes prohibitive, we may implement a pay-for-service fee, but it presently is supported by voluntary contributions from our users. It has worked extremely well and provided a vehicle for pushing out updates that affect the reliability and security of your server.

A Word About IPv6. Sooner or later Internet Protocol version 6 will be upon us because of the exhaustion of IPv4 IP addresses. Incredible PBX is IPv6-aware and IPtables has been configured to support it as well. As deployed, outbound IPv6 is not restricted. Inbound access is limited to localhost. You, of course, are free to modify it in any way desired. Be advised that disabling IPv6 localhost inbound access will block access to the FreePBX GUI. Don’t ask us how we know.

Originally published: Monday, August 11, 2014

Support Issues. With any application as sophisticated as firewall security, you’re bound to have questions. Blog comments are a terrible place to handle support issues although we welcome general comments about our articles and software. If you have particular support issues, we encourage you to get actively involved in the PBX in a Flash Forums. It’s the best Asterisk tech support site in the business, and it’s all free! Please have a look and post your support questions there. Unlike some forums, ours is extremely friendly and is supported by literally hundreds of Asterisk gurus and thousands of ordinary users just like you. You won’t have to wait long for an answer to your question.

Need help with Asterisk? Visit the PBX in a Flash Forum.

Some Recent Nerd Vittles Articles of Interest…

If you’re like us, spending $50 a month or more on a home security system is a bit like pouring money down the toilet. Add to that the complications of getting one to work reliably with VoIP without spending another $50 a month on a Ma Bell vintage telephone line just adds insult to injury.

So perhaps you can share our elation when an email arrived last week announcing Straight Talk’s new Remote Alert System, a $10/month cellular-based system that uses Verizon Wireless to provide SMS and phone call alerts for up to eight numbers. And actually it’s cheaper than that. $100 buys you a year of service. That’s less than $8.50 a month. Today we’ll show you how to transform your Prius-like Remote Alert System into a Tesla that will rival virtually any intrusion detection system on the market… at any price! The extra hardware required: any Asterisk-based server including the Raspberry Pi and BeagleBone Black.

Read and weep, ADT!

If we didn’t already have three Straight Talk lines of service, we would have filed this in the Too Good To Be True pile and moved on. But we’ve had terrific Almost-Unlimited™ AT&T Wireless service with Straight Talk for less than $500 a year. It’s not only indistinguishable from AT&T’s own offerings costing at least 50% more, but it’s also contract-free so we can bring any AT&T smartphone including iPhones to the party and never miss a beat.

We decided to take the bait and ordered the home security bundle. This gets you the Remote Alert wireless controller plus a wireless motion sensor plus a year of service for $229.99. If you prefer a one-month gamble, the bundle is only $139.99. Down the road, you can add additional motion sensors and window/door sensors for about $30 each. The add-ons now are available at Wal-Mart.

Shameless Plug. We obviously don’t charge for access to our articles. But you can assist the Nerd Vittles project financially by using our referral link with eBates® to make your purchase if you decide to try this. It doesn’t cost you a dime but returns 13.5% of your purchase price to the Nerd Vittles project. It’s just a couple of clicks. Start here to access eBates. Then Search for Straight Talk and click on the link. After the Straight Talk web site displays, click on the following link to access the Straight Talk Security Bundle. And, THANK YOU!

So… back to our story. The controller supports four zones for monitoring. Zone 4 is reserved for sensors you want to monitor while someone may still be moving around in the house, for example while only some of your family may be sleeping or if pets are roaming. The other three zones typically would be used for motion sensors that trigger alerts when anything moves… after giving you 30 seconds to leave and return, of course. You can activate Home or Away monitoring using either the controller, an optional $25 key fob, or a free app for your iPhone or Android smartphone.

You get to decide what happens when the system is armed and an alert is triggered either by motion or a monitored door or window being opened. For us, silence was the name of the game. Using the Android Remote Alert System, click the Silent ARM icon once you leave the house, and you’re done. When you return, click the Disarm icon within 30 seconds of opening the door, and monitoring is disabled. You can also enter your 4-digit alarm code on the controller to disable monitoring.

Remote Alert System Setup. Once you get the equipment, it’s a 5-minute phone call to get set up. Install the backup batteries in the controller and motion detector, and plug the controller into an A/C power source. Press the required sequence on the controller to activate it, and you’re in business. The motion detector is already paired with the controller when it arrives, but adding new sensors is a 15-second task. All of the commands are documented in the manual which accompanies the system. But the tutorials also are available on line if you want to have a look.

Step #1 is changing your security alarm password. The next step is entering your phone numbers. Straight Talk goes to great lengths warning you that this is not a home security system because it has no external siren and can’t make 911 calls. They obviously haven’t heard of Asterisk®. But let’s get through the standard setup before we talk about Asterisk integration. You get to set up three numbers to receive SMS text messages when an alarm is triggered. And you get to set up five phone numbers to receive calls when an alarm is triggered. What the called party will actually hear is an obnoxious alarm tone which continues to play for 15 seconds. If you had multiple properties with alarm systems and no Caller ID, you’d never know the source of the alarm! But people with multiple properties probably aren’t smart enough to use this system to begin with so let’s move on. You configure the SMS and phone numbers by entering a special code on the controller to program each of the eight destinations. Then you enter the 10-digit number twice, and you’re done. Easy Peasy!

If you’re new to home security systems, the key to motion sensors is placement. Straight Talk recommends placement about seven to ten feet off the floor with a wide field of view. The range of the motion sensor is about 26 feet. It obviously depends upon the layout of your house or apartment, but we had much better success placing the motion sensor on a window sill at about 5 feet high and aiming it at the center hall of our home. It improved the motion detection dramatically. Trial and error is your friend!

The next step is positioning your controller. A mounting bracket is included so that you can place it almost anywhere you like. Our preference is to hide it so long as it still has Verizon cellular coverage and a source of electricity. You can test it by arming the controller with your smartphone and then triggering the motion sensor. If you get an SMS message or a call, it’s working. We also prefer silent mode. An intruder is obviously going to attempt to destroy your controller if they hear it. Yes, the intruder may leave, but they’ll probably carry some of the family jewels with them. With an Asterisk server in place, we’d prefer to send the police without alerting the intruder that something has gone wrong.

Asterisk Integration. Speaking of Asterisk, here’s what we’ve developed to add 911 alerts and telephone alarms to this system. It’s a 5-10 minute project! The way this works is to first add a phone number to your controller that calls a dedicated DID on your Asterisk server. Calls to that DID trigger the special context [st-remote-alert] which verifies the CallerID number of your alarm system. As configured, if the CallerID doesn’t match, the call is immediately disconnected although you could easily modify our code to use an existing (non-dedicated) DID if you prefer. Just route the non-matching CallerIDs to whatever context you traditionally use to process inbound calls. If the CallerID of the alarm system is matched, then the call is disconnected AND an outbound call is placed to 911. When the 911 operator answers, a prerecorded message is played at least twice that says something like this using REAL information:

This is an automated security request for assistance from the residence at 36 Elm Street in Podunck, Arkansas. The owner of this residence is Joe Schmo at phone number: 678-123-8888. An intruder has been detected inside the home. A suspected burglary is in progress. All of the residents of the home are unavailable to place this call. Please send the police.

The phone number from which this automated call is being placed is 678-123-4567. If the owners have a working cell phone, you can reach them at the following number: 678-123-9999. Please dispatch the police to 36 Elm Street immediately, whether you can reach the owners or not.
A suspected burglary is in progress. Thank you for your assistance. This message will repeat until you hang up…

You can either use Flite and Igor to play the message, or you can record your own message to be played to 911. Use the FreePBX® Admin -> System Recordings option. We recommend the latter especially since you’ll be sending these emergency calls to 911. You obviously want the 911 operator to be able to quickly decipher what’s being said.

Legal Disclaimer. We cannot stress strongly enough that you need to test this carefully on your own server by placing test calls to some number other than 911 until you are positive that it is working reliably as determined solely by you. Be advised that this system will not work at all in the event of an electrical, Internet, or server outage. As delivered, this code will NOT place calls to 911. The choice of whether to modify the code to place 911 emergency calls is solely yours to make. Be advised that false and inadvertent calls to 911 may result in civil and criminal penalties. DON’T BLAME US!

NO WARRANTIES, EXPRESS OR IMPLIED, INCLUDING THE IMPLIED WARRANTY OF FITNESS
FOR A PARTICULAR PURPOSE AND MERCHANTABILITY, ARE BEING PROVIDED.

Asterisk Implementation. First, you’ll need a dedicated DID that can be used to receive incoming calls from your Remote Alert System. Hopefully, you won’t be receiving many calls on this number so any of the inexpensive pay-by-the-minute DIDs will suffice. Or you can use a free DID from ipkall.com. The only gotcha with ipkall.com is having to make a call to keep the number active at least once every 30 days. But this could be accomplished with a weekly telephone reminder that only connected for a few seconds. Just don’t make the weekly call using the CallerID of your alarm system. You obviously do not want to trigger a 911 emergency call.

Next, you’ll need an outbound trunk on your Asterisk server that’s previously been registered with E911 support and that already is configured to place outbound 911 calls from your server. Google Voice trunks will not work! Your name, address, and phone number as they were registered with E911 will be important pieces of information to relay in your automated emergency call to 911. You’ll also need a cellphone number that can be provided with your 911 calls so that emergency responders have a way to contact you to follow up on automated emergency calls from your server.

Temporarily, you’ll also need a 10-digit number to which to deliver the automated emergency calls for testing. Your cellphone number would suffice. Once you’re sure everything is working, we’ll show you how to modify the dial plan code to replace this number with 911 when your system goes “live.”

Installation. Once you have all of the required pieces in place, you’re ready to begin the installation. Log into your server as root and issue the following commands to begin:

cd /root
wget http://nerdvittles.com/wp-content/st-remote-alert.tar.gz
tar zxvf st-remote-alert.tar.gz
rm -f st-remote-alert.tar.gz
./st-remote-alert.sh

Once the install is finished, use FreePBX to modify the DID Trunk that will receive the incoming alerts from your Remote Alert System. Change the context entry to: context=st-remote-alert

Test. Test. Test. Testing is critically important before you actually turn on automated calls to 911. Once you’ve installed the software, activate your Remote Alarm System and then trip the motion detector to trigger a call to the dedicated DID on your Asterisk server. There’s typically a 30-second delay between tripping a motion detector and the commencement of the alert calls. Within a minute, you should receive a call on the emergency number you set up for testing. You can follow the progress of the procedure using the Asterisk CLI: asterisk -rvvvvvvvvvv. We recommend testing this repeatedly for at least a month before even considering 911 deployment. Make certain that everyone in your household knows how to disable the alarm system when they return home after arming it. Make certain that everyone in your household knows to never arm the system with motion detectors activated when anyone or any animal inside the house could potentially trip the alarm. At least until everyone is accustomed to these new security procedures and has a proven (successful) track record, NEVER DEPLOY SILENT ARMING OF YOUR REMOTE ALERT SYSTEM! If you change to silent arming of the Remote Alert System, test for at least another full month with no inadvertent failures before considering 911 deployment.

Making Changes. The st-remote-alert.sh installer has been designed to let you run it over and over again to replace or update your settings. So don’t be shy about making changes.

Substituting a Personally Recorded Message. If you’d prefer to record your own message to be delivered to 911, then review the script above and make yourself a cheat sheet before you begin. Then use a browser to open FreePBX. Choose Admin -> System Recordings and enter an extension number on your system to use for recording. Click the Go button to begin. Then dial *77 from that extension and record your message. Press # when you’re finished. Be sure to listen to the recording to make sure it’s what you intended. If not, rerecord the message until you get it right. You can dial *99 to listen to your recording a final time. When you’re sure it’s correct, name the recording nv-alert. Click Save.

Now you need to tell the automated alert dialer to use your recorded message instead of Flite and Igor.
Edit /etc/asterisk/extensions_custom.conf. Search for the line containing “pickrecording”. Change Extension: 4 to Extension: 5. Save the file and reload your dial plan: asterisk -rx "dialplan reload"

Do some additional testing if you have substituted your own recording!

Adding Audible Alarms During Emergencies. If you prefer a little noise sprinkled around your home during burglaries, then we’ve put in place the necessary components to sound alarms on SIP phones that support AutoAnswer after feeding an extension to the speakerphone. For example, assuming you have deployed a Yealink T46G with an IP address of 192.168.0.10 and default admin credentials, you could add this additional line just before the final s,n,Hangup line in the [st-remote-alert] context of /etc/asterisk/extensions_custom.conf:

exten => s,n,System(curl -s -S --user-agent "Alert" http://admin:admin@192.168.0.10/servlet?number=25276)

To add additional Yealink phones, just add additional lines to the dialplan with the IP address of each phone. For other phone models, you’ll need to do a little research.

Going Live with Automated Emergency Calls to 911. When you and everyone in your household are absolutely comfortable with the arming, disarming, and motion detection procedures, then you can decide whether to reroute the automated notifications to 911. Be advised that, in some states or municipalities, it may be illegal to auto-dial 911 from a non-human caller/system. Before doing this, check with an attorney or local authorities in your jurisdiction to make sure you are in compliance with federal/state/local laws.¹ If you elect to proceed, edit extensions_custom.conf in /etc/asterisk. Search for the line containing “SEND-HELP-REQUEST-TO”. Replace the temporary number that you set up with the number: 911. Save the file and reload your dial plan: asterisk -rx "dialplan reload". Sleep well!

Originally published: Monday, July 14, 2014

Support Issues. With any application as sophisticated as this one, you’re bound to have questions. Blog comments are a terrible place to handle support issues although we welcome general comments about our articles and software. If you have particular support issues, we encourage you to get actively involved in the PBX in a Flash Forums. It’s the best Asterisk tech support site in the business, and it’s all free! Please have a look and post your support questions there. Unlike some forums, ours is extremely friendly and is supported by literally hundreds of Asterisk gurus and thousands of users just like you. You won’t have to wait long for an answer to your question.

Need help with Asterisk? Visit the PBX in a Flash Forum.

Some Recent Nerd Vittles Articles of Interest…

1. Autodialers that make emergency calls to E911 as part of a burglar alarm system are specifically exempted in some states such as Illinois. This comports with federal law under The Telephone Consumer Protection Act (47 U.S.C. § 227). Emergency robocalls are specifically exempted from the new PSAP Do-Not-Call Registry rules. See also this article about E911 laws in the Northeast. In most cases, but not all, these laws target abuse of the E911 system. Surprisingly, one town that reportedly prohibits ALL autodialing to 911 is Palo Alto, CA. And Paris, Tennessee also has joined the illegal club. Special thanks to @TheMole on the PIAF Forum for his excellent research. [↩]

No. We’re not going to make you relive the 1970’s with us today although now you can listen to this Number 1 Hit and a million others for free with Amazon’s new Prime Music. No, we don’t get a commission if you sign up for Amazon Prime. Yes, we make millions when you buy something from Amazon using our links. Thank you! What we have for you today is a Number 1 Utility, and it works on virtually any Linux platform. If your fraternity or sorority had a secret knock to gain access, then you already know the basic concept. Port Knocker (aka knockd) from Judd Vinet is a terrific utility that runs as a daemon on your server and does just what you’d expect. It listens for knocks. When it detects three knocks on the correct three ports in the proper sequence and from the same IP address, it opens the IPtables Linux Firewall for remote access from that IP address to your server for a predefined period of time. This would allow you to log into your server with SSH or make SIP phone calls using a softphone registered to your remote Asterisk® server. What makes Port Knocker especially useful is the existence of knocking clients for virtually any smartphone, tablet, or desktop computer. For the Travelin’ Man, it’s another must have utility.

We introduced a turnkey implementation of Port Knocker in Incredible PBX for Ubuntu 14 late last week. If you were a pioneer earlier in the week, go back and install it again to take advantage of Port Knocker. Or better yet, follow along and we’ll show you how to install it on your own RedHat/CentOS or Ubuntu/Debian server in just a couple of minutes.

Prerequisites. We’ve built open source installation scripts for both the RedHat/CentOS platform as well as the Ubuntu/Debian operating systems. These knockd installers assume that you have a fully functional and locked down IPtables firewall with an existing WhiteList of authorized users. We’d recommend Travelin’ Man 3 if you need to deploy this technology and haven’t done so already. Last week’s Incredible PBX for Ubuntu 14 already includes Travelin’ Man 3 whitelisting technology. Read the article for full details.

Today’s knockd installers are fairly generic but, if you’re running a version of CentOS earlier than 6.x or Ubuntu earlier than 14 or Debian.anything, be advised that we haven’t tested these installers on those platforms so you’re on your own. Finally, if your server is sitting behind a hardware-based firewall (as we ALWAYS recommend), then you’ll also need to map the service you wish to access (e.g. UDP 5060 for SIP or TCP 22 for SSH) plus the three TCP ports from your hardware-based firewall to your server so that legitimate “knocks” can find their way to your server. The “knock” ports themselves do not need to be opened in your IPtables firewall configuration! We’re just knocking, not entering.

Overview. As configured, today’s installation scripts will install and preconfigure knockd to load automatically when you boot up your server. Three random TCP ports will be assigned for your server, and this port sequence is what remote users will need to have in order to gain access. Yes, you can change almost everything. How secure is it? Well, we’re randomizing the 3-port knock sequence using over 3,900 ports so you can do the math to figure out the odds of a bad guy guessing the correct sequence. HINT: 3900 x 3900 x 3900. Keep in mind that these “knocks” must all be received from the same IP address within a 15-second window. So sleep well but treat the port sequence just as if it were a password. It is! Once a successful knock sequence has been received, the default Port Knocker configuration will open all ports on your server for remote access from the knocking IP address for a period of one hour. During this time, “The Knocker” can log in using SSH or make SIP calls using trunks or extensions on the server. Port Knocker does not alleviate the need to have legitimate credentials to log into your server. It merely opens the door so that you can use them. At the bewitching (end of the) hour, all ports will be closed for this IP address unless “The Knocker” adds a whitelist entry for the IP address to IPtables during the open period. Yes, all of this can be modified to meet your individual requirements. For example, the setup could limit the range of ports available to “The Knocker.” Or the setup could leave the ports open indefinitely until another series of knocks were received telling knockd to close the IPtables connection. Or perhaps you would want to leave the ports open for a full day or a week instead of an hour. We’ll show you how to modify all of the settings.

Server Installation. To get started, log into your server as root and download and run the appropriate installer for your operating system platform.

For RedHat/Fedora/CentOS/ScientificLinux servers, issue the following commands:

cd /root
wget http://nerdvittles.com/wp-content/knock-R.tar.gz
tar zxvf knock*
rm knock-R.tar.gz
./knock*

For Ubuntu/Debian servers, issue the following commands:

cd /root
wget http://nerdvittles.com/wp-content/knock-U.tar.gz
tar zxvf knock*
rm knock-U.tar.gz
./knock*

For ARM-based servers, issue the following commands:

cd /root
wget http://nerdvittles.com/wp-content/knock-ARM.tar.gz
tar zxvf knock*
rm knock-ARM.tar.gz
./knock*

Server Navigation Guide. On both the RedHat/CentOS/Fedora and Ubuntu/Debian platforms, the knockd configuration is managed in /etc/knockd.conf. Before making changes, always shutdown knockd. Then make your changes. Then restart knockd. On RedHat systems, use service knockd stop and start. On Ubuntu, use /etc/init.d/knockd stop and start. By default, knockd monitors activity on eth0. If your setup is different, on Ubuntu, you’ll need to change the port in /etc/default/knockd: KNOCKD_OPTS="-i wlan0". On RedHat, the config file to modify is /etc/sysconfig/knockd and the syntax: OPTIONS="-i venet0:0".

In /etc/knockd.conf, create an additional context to either start or stop an activity. It can also be used do both as shown in the example code above. More examples here. There’s no reason these activities have to be limited to opening and closing the IPtables firewall ports. You could also use a knock sequence to turn on home lighting or a sprinkler system with the proper software on your server.

To change the knock ports, edit sequence. Both tcp and udp ports are supported. seq_timeout is the number of seconds knockd waits for the complete knock sequence before discarding what it’s already received. We’ve had better luck on more servers setting tcpflags=syn. start_command is the command to be executed when the sequence matches. cmd_timeout and stop_command tell knockd what to do after a certain number of seconds have elapsed since the start_command was initiated. If you’re only starting or stopping some activity (rather than both), use command instead of start_command and stop_command to specify the activity.

IPtables 101. The default setup gives complete server access to anyone that gets the knock right. That doesn’t mean they get in. In the PIAF World, it means they get rights equivalent to what someone else on your LAN would have, i.e. they can attempt to log in or they can use a browser to access FreePBX® provided they know the server’s root or FreePBX credentials.

If you would prefer to limit access to a single port or just a few ports, you can modify command or start_command and stop_command. Here are a few examples to get you started.

To open SSH access (TCP port 22):

/sbin/iptables -A INPUT -s %IP% -p tcp --dport 22 -j ACCEPT

To close SSH access (TCP port 22):

/sbin/iptables -D INPUT -s %IP% -p tcp --dport 22 -j ACCEPT

To open a range of SIP ports (UDP 5060 to 5069):

/sbin/iptables -A INPUT -s %IP% -p udp --dport 5060:5069 -j ACCEPT

To close a range of SIP ports (UDP 5060 to 5069):

/sbin/iptables -D INPUT -s %IP% -p udp --dport 5060:5069 -j ACCEPT

Here’s a gotcha to be aware of. If you’re using the Travelin’ Man 3 WhiteList setup on your server, be especially careful in crafting your IPtables rules so that you don’t accidentally remove an existing Travelin’ Man 3 rule in closing some port with knockd. You will note that the syntax of the knockd commands is intentionally a bit different than what you will find in your Travelin’ Man 3 setup. This avoids clobbering something accidentally.

Monitoring Activity. Here are the two best tools to monitor knockd activity to make certain your setup is performing as expected. The knockd log (/var/log/knockd.log) will tell you when a knocking attempt has occurred and whether it was successful:
[2014-07-06 14:44] starting up, listening on eth0
[2014-07-06 15:29] 79.299.148.11: opencloseSSH: Stage 1
[2014-07-06 15:29] 79.299.148.11: opencloseSSH: Stage 2
[2014-07-06 15:29] 79.299.148.11: opencloseSSH: Stage 3
[2014-07-06 15:29] 79.299.148.11: opencloseSSH: OPEN SESAME
[2014-07-06 15:29] opencloseSSH: running command: /sbin/iptables -A INPUT -s 79.299.148.11 -p tcp --dport 22 -j ACCEPT


Next, verify that the IPtables command did what it was supposed to do. iptables -nL will tell you whether port 22 access was, in fact, enabled for 79.299.148.11. The entry will appear just above the closing Chain entries in the listing:

ACCEPT     tcp  --  79.299.148.11         0.0.0.0/0           tcp dpt:22

Two things typically can go wrong. Either the knock from a client computer or cellphone wasn’t successful (knockd.log will tell you that) or IPtables didn’t open the port(s) requested in your knockd command (the iptables -nL query will show you that). In the latter case, it’s usually a syntax error in your knockd command. Or it could be the timing of the knocks. See /var/log/knockd.log.

Port Knocker Clients. The idea behind Port Knocker is to make remote access easy both for system administrators and end-users. From the end-user perspective, the simplest way to do that is to load an app on the end-user’s smartphone so that even a monkey could push a button to gain remote access to a server. If the end-user’s cellphone has WiFi connectivity sitting behind a firewall in a hotel somewhere, then executing a port knock from the smartphone should open up connectivity for any other devices in the hotel room including any notebook computers and tablets. All the devices typically will have the same public IP address, and this is the IP address that will be enabled with a successful knock from the smartphone.

Gotta love Apple’s search engine. Google, they’re not…

There actually are numerous port knocking clients for both Android and iOS devices. Here are two that we’ve tested that work: PortKnock for the iPhone and iPad is 99¢ and PortKnocker for Android is free. Some clients work better than others, and some don’t work at all or work only once. DroidKnocker always worked great the first time. Then it wouldn’t work again until the smartphone was restarted. KnockOnD for the iPhone, which is free, worked fine with our office-based server but wouldn’t work at all with a cloud-based server at RentPBX. With all the clients, we had better results particularly with cloud-based servers by changing the timing between knocks to 200 or 500 milliseconds. How and when the three knocks are sent seems to matter! Of all the clients on all the platforms, PortKnocker was the least temperamental and offered the most consistent results. And you can’t beat the price. A typical setup is to specify the address of the server and the 3 ports to be knocked. Make sure you have set the correct UDP/TCP option for each of the three knocks (the default setup uses 3 TCP ports), and make sure the IP address or FQDN for your server is correct.

Another alternative is to use nmap to send the knocks from a remote computer. The knock.FAQ file in your server’s /root directory will tell you the proper commands to send to successfully execute a connection with your server’s default Port Knocker setup. Enjoy!

Originally published: Monday, July 7, 2014

Support Issues. With any application as sophisticated as this one, you’re bound to have questions. Blog comments are a terrible place to handle support issues although we welcome general comments about our articles and software. If you have particular support issues, we encourage you to get actively involved in the PBX in a Flash Forums. It’s the best Asterisk tech support site in the business, and it’s all free! Please have a look and post your support questions there. Unlike some forums, ours is extremely friendly and is supported by literally hundreds of Asterisk gurus and thousands of users just like you. You won’t have to wait long for an answer to your question.

Need help with Asterisk? Visit the PBX in a Flash Forum.

Some Recent Nerd Vittles Articles of Interest…

We’ve devoted a lot of energy to Asterisk security over the years with our Primer on Avoiding the $100,000 Phone Bill and our 20 Failsafe Tips and our SIP Navigation Guide plus numerous tutorials on deployment of Virtual Private Networks to secure your servers and phones including NeoRouter, PPTP, and Easy OpenVPN among others. But, when it comes to ease of installation and use with rock-solid security, nothing comes close to deployment of WhiteLists with the IPtables Linux firewall that’s included at no cost with every major Linux distribution and with all of the Asterisk® aggregations including PBX in a Flash™ and Incredible PBX™. So we’re kicking off the summer with a careful look at the methodology behind IPtables and the Travelin’ Man™ tools developed to reduce the learning curve for new users.

Security, of course, is all about the “bundle of sticks.” As we learned from Aesop’s Fables, the more sticks you bundle together, the more difficult it is to break the stick. We are by no means advocating that you drop all of the other tools at your disposal to improve the security of your Asterisk security. So, before we dive into WhiteLists, let’s spend a little time covering some of the other tools that are available and why those tools should not be relied upon exclusively.

1. Hardware-based Firewall. The PBX in a Flash project has cautioned users for years not to run Asterisk-based servers connected to the Internet without a hardware-based firewall between your server and the public Internet. Is it failsafe? No. Some hardware-based firewalls have been compromised either by the bad guys or by the NSA. Pardon the redundancy. The other problem with hardware-based firewalls is that they’re generally not available with cloud-based solutions. As the price of cloud computing has dropped and the cost and headaches of maintaining your own hardware has increased, more and more folks are considering cloud-based alternatives. Yes. Hardware-based firewalls should be deployed whenever possible. No. They won’t resolve all security concerns.

2. Fail2Ban. Once upon a time, a number of us thought that Fail2Ban was the answer to all security issues with Asterisk-based servers. In a nutshell, Fail2Ban scans your logs searching for failed attempts to log in to either SSH, FTP, Apache, SIP, or an email account. After a small number of failed attempts, Fail2Ban blocks further access from the IP address initiating the requests. There are two problems with Fail2Ban. First, software developers of the affected services continue to “improve” things with new and different error messages when login failures occur. Since Fail2Ban is searching for specific word matches to identify unsuccessful logins, the whole security mechanism fails when the “magic words” change unless everyone is extremely vigilant in maintaining the “magic word” lists AND updating the Fail2Ban rules on all of your servers. Our experience suggests that the bad guys find the new “magic words” long before everyone else which means there are gaping holes in Fail2Ban regularly. The other problem is supercomputers such as Amazon EC2 which makes enormous computing resources available to every Tom, Dick, and Harry. We’re mostly worried about the Dick that can hammer your little server every second with hundreds of thousands of attempts to crack your SIP or SSH passwords. The problem this poses is that most Linux servers never allocate a sufficient time slice to Fail2Ban to scan your Asterisk, Apache, and SendMail logs. Instead of blocking a bad guy after 3 failed login attempts, a bad guy using EC2 may be able to perform several hundred thousand login attempts before Fail2Ban ever detects a problem. Yes. Fail2Ban helps against the bad guy manually keying in passwords. No. Fail2Ban is all but worthless against a sophisticated denial of service attack on your server.

3. Virtual Private Networks. The beauty of virtual private networks (VPNs) is that all of your Internet traffic is encrypted and tunneled through private IP addresses that others can’t intercept. That was the theory until Edward Snowden came along and spoiled the NSA’s party. Yes. We’ve known that PPTP VPNs were vulnerable for a good long while. No. We didn’t know that the NSA (and presumably others) may have had the keys to your castle much longer… regardless of the VPN topology you may be using. The other problem with VPNs is that you need VPN connections for every device connecting to your server. Unfortunately, VPN technology is only available on a small number of SIP telephones, and the supported OpenVPN topology is one of the more difficult VPNs to deploy on a Linux server. Are VPNs better than nothing? Absolutely. Does a VPN provide failsafe communications security over the open Internet? Probably not.

4. Nothing Beats Secure Passwords. Amen. There was a time when some Asterisk-based servers were routinely set up with extension passwords of 1234 or the extension number itself. And outbound SIP trunks were deployed with no dialing rules. And administrators opened accounts with SIP providers with automatic credit card replenishment whenever the accounts ran out of money to cover calls. And no safeguards were put in place to restrict international calling. Little did these folks know that registering to a SIP extension on an Asterisk server provided a blank check for making unlimited calls to anywhere on the planet. Thus was born the $100,000 phone bill. Yes. Nothing Beats Secure Passwords for root, for SIP accounts, and for SIP and IAX trunks connected to commercial providers. But you also need to implement dialing rules for outbound calls that allow your callers to reach only the destinations desired, not the world. And your accounts with providers should always include limits and restrictions on international calls and should never include automatic credit card replenishment.

5. BlackLists. There was a time when blacklisting IP addresses was believed to be the ultimate solution to Internet security problems. Sounds great, doesn’t it? Just set up a database with the IP addresses of all the bad guys in the world, and all our problems will be solved. Problem #1: A new bad guy is born every minute. Problem #2: The bad guys learned how to use VPNs and other random IP address masquerading sites to disguise their true identity. Problem #3: Security vulnerabilities in many Windows-based machines allowed the bad guys to take control of these computers and do their dirty work from there. Problem #4: There are actually some good guys that live in Russia and China. Problem #5: The bad guys learned to poison the “bad guy list” to block essential services such as DNS, Google, Amazon, Netflix, Pandora, and your favorite bank and credit card companies. Yes. The theory of blacklists sounded great. No. Blacklists not only don’t work. They’re downright dangerous.

For the past few years, our Internet security focus has turned toward defining a methodology that works with all PBX in a Flash and Incredible PBX servers, whether they’re dedicated servers behind a hardware-based firewall or public on a cloud-based shared host. And the conclusion we’ve reached is that nothing beats the IPtables Linux firewall for rock-solid Internet security. The reason is its deep integration into the Linux kernel itself through Netfilter, “a set of hooks inside the Linux kernel that allows kernel modules to register callback functions with the network stack.” Wikipedia provides an excellent overview for those with an interest. For our purposes, suffice it to say that IPtables examines inbound and outbound packets before any further processing occurs on your server. With our default setup, we typically allow all outbound traffic from your server. For inbound traffic, if the iptables rules permit access, the packet comes in for processing. If not, the packet dies at the door with no acknowledgement that it was even received. In laymen’s terms, if someone attempts to scan your server to determine whether web or SIP services are available, there will be no response at all unless packets from the scanning server’s IP address are permitted in the iptables rules configured on your server. You can determine which rules are in force with this command: iptables -nL.

The basic configuration and syntax of iptables rules can be daunting to those unfamiliar with the territory. And thus was born Travelin’ Man 3, our open source tool to simplify configuration of IPtables by allowing administrators to define WhiteList entries describing the types of services that were allowed access to a server from specified external IP addresses. The basic rules of the Travelin’ Man 3 setup for iptables are these: (1) outbound packets are unrestricted, (2) forwarded, established, and related packets are permitted, (3) inbound packets from the private LAN are unrestricted, but (4) inbound packets from the public Internet are dropped unless permitted by a specific iptables rule. Those rules include certain basic services such as time synchronization (TCP 123) as well as WhiteListed IP address entries for specific or generic services.

Installation is easy. Log into your PBX in a Flash as root and issue the following commands. NOTE: Travelin’ Man 3 is optionally available as part of Incredible PBX installs on the CentOS, Scientific Linux, and PIAF OS platforms. It is preinstalled on the Raspberry Pi and BeagleBone Black platforms with RasPBX. You can determine if it’s already installed on your server with this command: ls /root/secure-iptables. If the script exists, you’ve already got Travelin’ Man installed, but it may not be running so keep reading…

cd /root
wget http://incrediblepbx.com/travelinman3.tar.gz
tar zxvf travelinman3.tar.gz
yum -y install bind-utils
./secure-iptables


Because PBX in a Flash and Incredible PBX servers are primarily designed to support telephony, Travelin’ Man 3 further simplifies the iptables setup by whitelisting the IP addresses of a number of the leading VoIP providers. These include Vitelity (outbound1.vitelity.net and inbound1.vitelity.net), Google Voice (talk.google.com), VoIP.ms (city.voip.ms), DIDforsale (209.216.2.211), CallCentric (callcentric.com), and also VoIPStreet.com (chi-out.voipstreet.com plus chi-in.voipstreet.com), Les.net (did.voip.les.net), Future-Nine, AxVoice (magnum.axvoice.com), SIP2SIP (proxy.sipthor.net), VoIPMyWay (sip.voipwelcome.com), Obivoice/Vestalink (sms.intelafone.com), Teliax, and IPkall. For the complete list: cat /etc/sysconfig/iptables (CentOS) or cat /etc/network/iptables (RasPBX).

The real beauty of Travelin’ Man 3 is you aren’t limited to our WhiteList. You can add your own entries easily using the TM3 scripts that are included in the /root directory. secure-iptables initializes your iptables setup and also lets you define a primary IP address or fully-qualified domain name (FQDN) that will always have access to your server. You must run this script at least once to activate IPtables on all platforms!

Once you have run secure-iptables, you can whitelist additional IP addresses by running add-ip. You can whitelist additional FQDNs by running add-fqdn. You can delete either IP addresses or FQDNs by running del-acct. As noted previously, you can check what’s authorized with the command: iptables -nL.

We’ve also included a custom script to restart IPtables gracefully: iptables-restart. The reason is because using the traditional restarting mechanism in IPtables will leave your server vulnerable (and IPtables inoperative) if a particular FQDN cannot be resolved. The iptables-restart script takes another approach and removes the offending rule from your whitelist, alerts you to the problem, and then restarts iptables without the offending entry. So all existing rules are put back in place and function as you would expect.

Finally, Travelin’ Man 3 includes a script that allows you to utilize FQDNs for users that may have ever-changing dynamic IP addresses. Steps #4, #5, and #6 in the original Travelin’ Man 3 tutorial will walk you through the Administrator set up which only takes a minute or two and never has to be touched again. Basically, a cron job script is employed to check for changes in the dynamic IP addresses you have identified with FQDNs. If changes are found, IPtables is restarted which updates the IP addresses accordingly.

Unfortunately, there was one group of end-users that weren’t covered by the Travelin’ Man 3 setup. This group included traveling salespeople or vacationing individuals that may land in a different city every night. Rather than relying upon an administrator to provide access to home base, these frequent travelers needed their own tool to manage their IP address as it changed. While this was supported through a web interface in Travelin’ Man 2, that setup exposed your web server to the public Internet and was burdensome for administrators to initially configure. Most importantly, it didn’t manage remote IP address access using IPtables which made coexistence with TM3 difficult. Thus was born Travelin’ Man 4.

Travelin’ Man 4 is a new add-on for an existing Travelin’ Man 3 setup. It’s for those that wish to allow traveling individuals to manage their own whitelist access to PBX in a Flash or Incredible PBX using a telephone. An Administrator preconfigures accounts and passwords for the travelers together with the services to which they will have access on the server. Using any cellphone or hotel phone, the traveler simply dials a preconfigured number to access an IVR that will prompt the user for an account number and PIN. Unless you have a spare DID, you can grab a free one from IPkall.com to use with your Travelin’ Man 4 IVR. Once a user is successfully logged in, the IVR will prompt for the user’s IP address to be whitelisted on the server. Enter it using this format: 12*34*56*78.

Within a couple minutes, the new IP address will be properly formatted and then whitelisted in IPtables, and the traveler will be sent an email acknowledging that the account has been activated. Once the account is activated, the traveler can use a SIP softphone application such as Zoiper on any iPhone or Android phone or a softphone on any desktop computer to place and receive calls as well as to check voicemail on the remote PBX in a Flash server. For anyone that doesn’t know their current IP address, a quick visit to WhatIsMyIP.com will tell you. Travelin’ Man 4 is licensed under GPL2 so download a free copy. Then read the tutorial and give it a whirl. Enjoy!

Originally published: Wednesday, May 21, 2014

Need help with Asterisk? Visit the PBX in a Flash Forum.

Some Recent Nerd Vittles Articles of Interest…

Grandstream has done with Asterisk what Samsung and others did with Android. You basically take a freely available, open source toolkit and transform it into a terrific piece of turnkey hardware with tremendous savings in development costs. While it’s great for consumers, to us it highlights what is wrong with the GPL2 license which lets companies do this in the first place. These for-profit companies give almost nothing back to the open source community. Remember, it’s not their toolkit which took talented (and uncompensated) developers hundreds of man-years to construct. In Samsung’s case, they built closed source smartphones and tablets. With the Grandstream UCM6100 series, you get closed source PBXs. What’s wrong with this picture? Lots! You’re taking someone else’s work product, embellishing it to make a profit, and returning nothing to the open source community that made your open source product possible in the first place. Don’t get us wrong! We love Samsung’s smartphones and tablets. We’ve owned at least a half dozen of them. And Grandstream’s UCM6100 is an incredibly useful appliance for home offices as well as small and large organizations. We can think of a thousand use cases for the UCM6100 in the corporate and government workplace. If done right, it could easily have replaced the $200,000 PBX that supported 100+ employees in one of my former organizations. We also should note that Grandstream isn’t the first company to attempt this feat with Asterisk. Read Tom Keating’s excellent article for the history. And don’t forget the AA50 for a few cents more.

What is disappointing is that all of these products would be so much better and so much safer if the companies would open source their code and encourage community development to finish the job they started.¹ No individual and few companies could match the hardware development platform that Samsung and Grandstream have managed to put together. In Grandstream’s case, you can buy the UCM6102 at retail for $264! It includes two FXS ports for devices such as fax machines and two FXO ports for interconnecting your Ma Bell PSTN trunks to a one-pound SIP powerhouse. That $264 buys you an incredibly attractive piece of hardware with an LCD that tells you everything about your PBX at the click of a button. And there are small LEDs to display the status of the LAN, WAN, USB, SD card, Phone, Fax, and both Telco lines. The device can sit under your phone on your desk in a SOHO office, or it can be wall-mounted in the closet of a bank’s branch office. Models are also available with 4 FXO ports (pictured above) as well as 8 and 16 FXO ports. One of these could meet the needs of almost any organization, regardless of size. Amazing hardware technology, really!

The web-based software user interface (UI) is no less impressive. FreePBX® has been our development partner on open source Asterisk® projects for the better part of a decade. To say they’ve made Asterisk what it is today is an understatement. Asterisk is a toolkit. FreePBX makes it a useful PBX for millions of users around the globe. Having said all of that, competition makes the world go ’round. And Grandstream has built an impressive UI for the UCM6100 devices. What is more amazing is to compare the performance of the Grandstream device to our own Incredible PBX for the Raspberry Pi which runs with Asterisk and FreePBX on a virtually identical processor with the same memory constraints as the UCM6100 devices. Night and day is the only way to sum it up. The Grandstream PBX literally runs circles around the Raspberry Pi in hardware and UI performance. In fact, you would never know the Grandstream PBX wasn’t running on a quad-core processor with several gigs of RAM if you were judging by performance. And there’s even a little fan that comes on about once an hour as if to remind you that there’s a real computer under the covers.

After receiving our UCM6102 late last week, we put it through its paces. We set up extensions and trunks and ring groups and outbound routes and inbound routes. We tested voicemail. We configured an IVR. We uploaded custom voice prompts. We tried out the Parking Lot and Call Forwarding and Conferencing. It all worked swimmingly, and configuration took only minutes with the web-based UI which was quite intuitive given its similarity to older releases of FreePBX such as 2.8 and 2.9.

But, in the words of Geoffrey Chaucer, “All good things must come to an end.” Our next mission was to interconnect the UCM PBX with one of our existing PBX in a Flash servers. After all, the real utility of a turnkey PBX appliance like this would be to support a branch office with no technical staff in residence. This would allow a bank or a hospital or a real estate company to interconnect sites with extensions at each site that could transparently connect to each other. For example, dialing 5000-5099 would ring phones in the main headquarters while dialing 5300-5399 would ring phones in branch office #3. For this to work in the Asterisk environment, we need password-protected trunks on each Asterisk server that interconnect the PBXs to each other to form a meshed network. It’s not difficult, and we’ve explained how to do it in previous Nerd Vittles articles using PBX in a Flash as well as Incredible PBX for the Raspberry Pi.

Trunk to Trunk Server Connections. As the screenshot above shows, connecting a trunk from the Grandstream PBX to our Asterisk server was a breeze using both SIP and IAX trunks. But attempts to connect a trunk from the Asterisk server to the Grandstream PBX using both SIP and IAX failed with password errors. When we alerted the Grandstream development team, suffice it to say they were confused. Did we mean we wanted to connect a remote Asterisk server to an extension on the UCM6100? That was the first hint that all was not well in Asterisk Land. It became readily apparent that the developers were quite adept at mimicking the functionality of FreePBX to create a powerful PBX. But they lacked an in depth understanding of some of the Asterisk fundamentals. While the Grandstream development team was incredibly responsive, it reinforces why open sourcing their code would provide huge benefits not only to others but also to their own project. It gets worse, unfortunately, much worse.

To make a long story short, it doesn’t appear that safely interconnecting trunks between Asterisk servers and the Grandstream devices is available at least at this juncture. What is possible and what the Grandstream developers documented is the ability to create a trunk on a remote Asterisk server that registers to an extension on the Grandstream PBX. But this still did not enable users on remote Asterisk servers to call extensions on the Grandstream PBX unless the Allow Guest Calls option was enabled in the device’s SIP settings. That didn’t make a lot of sense to us if, in fact, the remote Asterisk server was actually registered to the Grandstream PBX. So we changed the password on the extension to make sure the registration would fail. And, yes, you still could make calls to the Grandstream PBX extensions so long as Allow Guest Calls was enabled. Did we mention? It gets worse, much worse.

IVR Vulnerability. Remember that IVR setup we mentioned? By default, it sits on extension 7000 on the Grandstream PBX. We called it from an extension on the remote Asterisk server, and it worked as expected even without a valid SIP registration so long as Allow Guest Calls was enabled. You probably can guess what our next test was. We disabled Allow Guest calls and attempted to call an extension on the Grandstream PBX. It rang busy as it should. We then dialed extension 7000, and guess what? The call went through. Whoa! Remember, SIP guest calls had been disabled, and there was no SIP registration because of a password mismatch. In short, anybody from anywhere that knew the public IP address of our Grandstream PBX could now connect to any IVR on the device just by knowing that the IVRs begin with extension 7000. It’s a classic dial plan mistake of letting external calls bleed into privileges which should be reserved for internal users. For security and other reasons, it’s also why FreePBX does not assign extension numbers to IVRs. But there’s more.

Stealth AutoAttendant Gone Bad. As you can see from the IVR Setup screen shown above, two of the options available when setting up an IVR are to enable calls to Extensions and to Trunks. Many administrators as well as casual users that barely understand what they’re doing probably would enable these features believing the options would be restricted to local use by the default guest call restriction. Wrong! What it means in terms of this security lapse is that now any anonymous caller with your IP address can dial into your Grandstream PBX and, while the IVR announcement on the default IVR extension (7000) is playing, the anonymous caller can dial any Extension or any long distance call supported by the Grandstream PBX trunk configuration so long as these options were enabled in the IVR. In Nerd Vittles parlance, think of it as a remake of our Stealth AutoAttendant with Public DISA Connectivity… for the world!

FXO/PSTN Warning. In discussing this with Tony Lewis of Schmooze and FreePBX fame, he reminded me that we’re talking about a PBX that’s been designed for business use with FXO ports and PSTN trunks. So, while the SIP vulnerability at least required that someone know the IP address of your PBX, once you connect PSTN lines to the Grandstream PBX and answer incoming calls with an IVR on the system, all bets are off. Anonymous bad guys now can place PSTN calls to any published phone number for your server that happens to connect to an IVR. These calls then can be used as the springboard to place outbound calls to anywhere the PBX trunk setup permits. Get out your checkbook!

Other Security Issues. Having owned the device for only a few days, we obviously have not tested all of the potential attack vectors. There are other anomalies in the dial plan code which we really can’t quite figure out without seeing the actual code. We were going to try to document an equally serious issue with the trunk peering, but your head would probably explode just trying to wrap your head around the problem. Ours did! Suffice it to say, with a single outbound route to a registered trunk that has failed to register, all outbound calls initiated by internal and external callers should always fail. They don’t! We’re also unclear whether the appliance provides SSH access for the root user. In any case, you aren’t provided the password. That could potentially be a problem if, in fact, a root account is enabled on the appliance. Finally, we should note that, according to the GPL materials published by Grandstream, this appliance is running Asterisk 1.8.9.3. Twenty-five versions of Asterisk 1.8 have been released since that offering appeared eight months ago. Some of those updates patched serious security vulnerabilities in the Asterisk 1.8 code.

Until Grandstream addresses some of these security issues, you are well advised to only operate a Grandstream PBX behind a secure, hardware-based firewall with no Internet port exposure. We would caution against connecting PSTN trunks to the device at this juncture. If you’re feeling lucky, a possible option for the time being would be to disable IVRs and especially the extension and trunk dialing options. That alternative unfortunately defeats the real purpose of buying these devices.

I Have A Dream. Not to beat a dead horse, but discoveries like this reinforce the need for companies such as Grandstream to revisit their design strategy and give serious consideration to open sourcing their code. After all, Grandstream is primarily a hardware company, and they could sell a gazillion of these appliances if the platform were open. We’ve hurriedly compiled a list of features that currently are missing which could be added almost overnight if this were an open source project. The PBX in a Flash development team would be at the front of the line to assist!

1. No text-to-speech functionality
2. No speech-to-text functionality
3. No (intended) DISA functionality (but data is collected in syslog??)
4. No ability to load custom dialplan code
5. No AGI/PHP script support
6. No Google Voice support for free calling in U.S. and Canada (add it for $30 like this)
7. No SIP/IAX trunk registrations from remote Asterisk servers
8. No incoming calls except via anonymous SIP or PSTN (nixes interoffice setups for extensions)
9. No traditional fax support except using fax machine on FXS port (T.38 is supported)
10. No access to Asterisk CLI for debugging or otherwise
11. Crippled SSH access (basic config info, set/get variable, upgrade, reboot, reformat)
12. No VPN support
13. No SIP security with Internet exposure
14. No Fail2Ban support
15. No WhiteList security to lock down the server

Recommendations. In closing, we don’t mean to suggest that security vulnerabilities never occur in open source code, but open source does guarantee that hundreds if not thousands of developers would be reviewing the code rather than a handful of people that may not fully appreciate all of the nuances of Asterisk. And each time a discovery like this occurs that has the potential of costing unsuspecting companies thousands of dollars in unanticipated phone bills, it gives Asterisk an undeserved black eye. Issuing a patch unfortunately won’t cure this problem for most purchasers because most purchasers never upgrade firmware on appliances.

We hope Grandstream will either pull the devices from the marketplace until the default firmware is fixed or place a big orange warning sticker on the boxes warning purchasers to upgrade the firmware and explaining the consequences of not doing so. Better yet, do the right thing and open source the platform and the code so that others can benefit from Grandstream’s development work on what still could be an incredibly useful and amazing device.

July 31 Update: After an exchange of emails with Grandstream, we have a better understanding of their call routing methodology that we want to pass along. It should be noted that the security holes we documented still exist, but there are mechanisms in place to stop the bleeding… if you know how to use them. Grandstream relies upon a set of Privilege Levels for extensions and IVRs as well as inbound and outbound routes. These include Internal, Local, National, and International. Only Extensions and IVRs with matching or higher privileges can use Inbound and Outbound Routes of a matching or lower privilege level. Read that again! It’s important. For example, if an extension has Internal privileges (the default), then that Extension can only access Outbound Routes designated as Internal. Calls to other numbers will fail. Unfortunately, all routes default to Internal, and this security mechanism is barely documented in the User Manual. Unlike FreePBX which uses Outbound Routes to connote calls leaving your server, Outbound Routes in Grandstream parlance are a set of dialplan rules for every call. Stated differently, to have a secure system, you need to create an Outbound Route for every possible type of external AND internal call. The same holds for Inbound Routes. Here’s an example of how to safely configure Trunks and Extensions between the Grandstream PBX and a remote Asterisk server so that extension-to-extension calls can be made between the two offices while insulating your IVRs from the long distance free for all that we documented in the original article.

Unfortunately, the IVR setup is still buggy and hence vulnerable. As the chart at the end of this article makes clear, there presently is no way to configure an IVR in such a way that remote callers cannot make long distance trunk calls while local extensions can. The only options presently available are either to disable the Dial Trunk option or to set the IVR Privileges lower than the Privileges setting for your outbound trunks. Do NOT rely upon a separate IVR for local users with the Dial Trunk option enabled thinking you’re safe. You’re not! Our original article above explains the possible consequences.

Remote Asterisk Server Setup Using FreePBX. On our remote server, we want to create two Trunks and an Outbound Route. One trunk will be used to set up an outbound registration to an Extension on the Grandstream PBX. We’ll use this trunk to place calls to Grandstream PBX extensions, IVRs, and conference rooms. The other trunk will be used to authenticate an inbound registration from the Grandstream PBX. The Grandstream PBX extensions will use this trunk (with registration from the Grandstream PBX) to initiate calls to extensions registered on our remote server. The outbound route will be used to route calls using the outbound registration trunk to Grandstream PBX extensions, IVRs, and conference rooms.

Here is the outbound registration trunk to extension 5001 on the Grandstream PBX (192.168.0.120 in our example):

Here is the inbound registration trunk to authenticate the Grandstream PBX matching trunk:

Here is the outbound route that allows extensions on the remote server to call Grandstream extensions, IVRs, and Conference Rooms:

You would also want to create an Inbound Route for 5001 that sends incoming calls from dialing 5001 on a Grandstream PBX extension to a particular destination on your remote server. Otherwise, the calls would be processed using the FreePBX default inbound route if you happen to have one. In our setups, we typically point the default inbound route to an IVR or a receptionist’s extension.

Grandstream PBX Setup to Connect to Remote Asterisk Server. To make all of this work securely, we need to create an Extension to handle the inbound registration from the remote Asterisk server so that users on the remote server can call extensions, IVRs, and conference rooms on the Grandstream PBX. And we need a SIP trunk that will register to the remote Asterisk server so that Grandstream PBX users can call extensions on the remote Asterisk server. Then we need Inbound and Outbound Routes to lock things down. We’re using 192.168.0.181 as the IP address of the remote Asterisk server in this example. The key point in securing the Grandstream PBX is to assign the proper permissions to the Grandstream Extension and IVRs that will be used with remote server connections. Then elevate permissions where necessary on the Inbound and Outbound Routes to make sure only our truly local extensions can make calls using Grandstream long distance and PSTN trunks. Don’t confuse local extensions with Local permissions. A local extension is an extension that registers to the Grandstream PBX. Local permissions is a security level that means a particular resource can only do things with other matching Internal or Local resources and with no resources that have been assigned a higher permission level. Internal permissions means a resource can only do things with other Internal resources. Clear as mud? We know. Hang in there until we’re finished.

First, create extension 5001 that will be used by the remote Asterisk server to register with the Grandstream PBX:

Next, create a SIP Trunk that will register to the remote Asterisk server at 192.168.0.181. We’ve used 1234 as the password in our examples so plug that in for the time being. You obviously would want something more secure than that! You’ll note that you don’t assign a Permission level to a Trunk. That is handled in the Inbound and Outbound Routes which tie particular routes to designated trunks. So Trunks inherit their permissions based upon a matching route. We suspect this may be the root cause of the security holes that we’ve documented. If there is no specified route for a particular type of call, Grandstream is doing something internally to make a determination on whether to allow the call or not. In some cases, that determination just happened to be wrong.

For truly local users, i.e. extensions directly connected to the Grandstream PBX, you need to elevate the Permissions for those extensions to reflect the types of calls you want them to be able to make. Typical permission for these extensions would be National or International. The same holds true for IVRs. Elevate IVR permissions to restrict usage to your intended audience. Keep in mind that we’re treating calls to extension 5001 on the remote Asterisk server as Internal. That’s the bottom rung in the security ladder which means every local extension and IVR will be able to place calls to that extension. If this isn’t what you want, then you’ll need to elevate the 5001 extension permissions accordingly. For example, you may only want Grandstream PBX extensions with Local call permissions to be able to call extensions on the remote PBX. In this case, you would want to change the 5001 extension permission level to Local.

Let’s tackle the Inbound Routes next since this was the cause of the inability to connect to local Grandstream extensions from the remote server. If you’re using the default Grandstream setup, then you’ll need Inbound Routes for both _50XX extensions and _70XX IVRs to permit remote callers to connect with Grandstream PBX extensions and IVRs with Local permissions only. This means that even if they connect to the 7000 IVR, they will not be able to make long distance calls on your nickel even if Trunk dialing is enabled.

The Inbound Route rule for Extensions should look like this:

The Inbound Route rule for your IVRs should look like this:

The key point to keep in mind with Inbound Route IVR permissions is to keep the permission level LOWER than whatever permission level you assign to the Outbound Route for placing calls that cost you money, typically National and International.

Now let’s set up the Outbound Route to restrict outbound calls to 10-digit numbers for extensions, IVRs, and Inbound Routes to those with at least National permissions. Keep in mind you may need additional outbound routes with Local permissions for certain 10-digit numbers if your local calling area happens to include free calling to multiple area codes, e.g. Atlanta.

Depending upon your setup, you may need additional dialplan rules and outbound routes to handle 11-digit numbers which should be routed out through a PSTN trunk, e.g. 1NXXNXXXXXX. And because of the security hole, be sure to add a catch-all for international calls that requires International permissions. The dial string XXXXXXXXXXX. will catch everything not included in the NXXNXXXXXX and 1NXXNXXXXXX outbound rules.

Finally, you’ll need an Outbound Route that allows local callers on the Grandstream PBX to connect to extensions on the remote PBX. You typically would assign Internal or Local permissions to this route which would look something like the following depending upon the extension configuration on your remote PBX:

A Word of Caution on IVRs: In the Grandstream security model, IVRs have their own Privilege levels. At least at this juncture, that Privilege level can “promote” the permissions of a call that began at a lesser privilege level. For example, if your Inbound Route for 7XXX calls is assigned Local privileges and the 7000 IVR is assigned National privileges, an incoming call to 7000 from a remote PBX will “inherit” the National privileges of the IVR. This obviously should never be possible. Either the 7000 IVR should generate Congestion and not answer the call at all where the Inbound Route has lesser privileges than the IVR. Or, at the very least, those options in the IVR (including stealth extension and trunk dialing) that require National or International privileges should generate Congestion and disconnect the call. For the time being, ALWAYS set the Privilege level of an IVR to the lowest permission threshold to protect your server and wallet from the consequences of placing unintended toll calls. Here’s a little chart we put together to document the impact of merely changing the Privilege setting for the 7000 IVR:

Other Tips and Tricks. Here are a few other suggestions to expand the functionality of your Grandstream PBX:

Add Google Voice Support with an OBi Device

Add Bluetooth Cellphone Trunk with an OBi202

Add Free iNum Calling Worldwide with a VoIP.ms Account using an OBi202

Continue reading Part 2…

Deals of the Week. There are a couple of amazing deals still on the street, but you’d better hurry. First, for new customers, Sangoma is offering a board of your choice from a very impressive list at 75% off. For details, see this thread on the PIAF Forum. Second, a new company called Copy.com is offering 20GB of free cloud storage with no restrictions on file size uploads (which are all too common with other free offers). Copy.com has free sync apps for Windows, Macs, and Linux systems. To take advantage of the offer, just click on our referral link here. We get 5GB of extra storage, too, which will help avoid another PIAF Forum disaster.

Originally published: Tuesday, July 30, 2013

Need help with Asterisk? Visit the PBX in a Flash Forum.

Don’t miss the first-ever FreePBX World on August 27-28 at the Mandalay Bay in Las Vegas. For complete details, see this post on the FreePBX blog.

Some Recent Nerd Vittles Articles of Interest…

1. It turns out Grandstream may not have much of a choice but to open source their code. It now appears their PBX and User Interface are both based upon open source GPL2 software owned by Digium. [↩]

UPDATE: Following a week of almost daily new disclosures, we are republishing our original article with some additional observations from us and others who have been following what will almost certainly turn out to be one of the most shocking and important revelations in the history of our republic. Much of this story has unfolded on Twitter, one of the few companies that stood up to the government and refused to participate in the PRISM data collection enterprise. So it seemed only fitting to retell the events of the last few weeks through the tweets that have brought this story to life and fleshed out many of the details pertaining to Mr. Snowden’s original revelations. –Ward Mundy

We were up to our eyeballs in Vietnam when I graduated so my career path was chosen for me. Those with political connections and those from well to do families got to join the National Guard and stay home. The rest of us got drafted. If you were a lawyer, you had the choice of serving two years as an infantry officer or four years as a lawyer. That was an easy choice at least for me because I owed the military four years anyway because of a college scholarship. There was a great book at the time, Military Justice is to Justice as Military Music is to Music. You really didn’t have to read the book to figure out the message. As an appellate lawyer, I got to witness it up close and personal. Military courts have special rules. Military commanders decide who gets tried and then they hand-pick the military juries who also happen to be soldiers working for the same commander. If a criminal case went to trial, your odds of not being convicted were about the same as being struck by lightening or winning the lottery. The same held true for the criminal appeals. All of my clients were already serving time in Fort Leavenworth while I “represented” them in Washington. A handful saw their sentences reduced on appeal while one or two actually had their cases reversed. Despite the odds favoring the military, I actually worked for an appellate judge that would hide problematic cases in a bottom drawer until the soldier had served all his jail time. Then the case would magically reappear, and justice would be done by reversing the errors in the trial proceedings. Several of us finally filed a complaint against the judge, and he was “retired.” This was my first exposure to the “Rewards System.” Many folks are promoted into prestigious jobs in the government not to do future good work but as a reward for past service. In short, it’s treated more like a medal than a job. Suffice it to say, you don’t see a lot of boat rocking from the honorees.

At the time, I chalked this up as yet another military anomaly. It was reinforced by the decade I spent at all levels of the military justice system including a stint as the Court Executive of the U.S. Court of Military Appeals, the civilian court that oversaw the whole process. What was surprising to me were the extremes to which many individuals in the most important legal positions in the military were willing to go in order to rig the system in favor of what they perceived to be the desires of the military commanders. My dad had been a military commander. He and all of the commanding generals I have known would never have tolerated such a setup had they known about it. While the President technically appointed judges to the Court of Military Appeals, in actuality the judges were selected by members of the Senate Armed Services Committee. Because the Court had become too liberal for some of the senior people in Donald Rumsfeld’s Defense Department, they decided to change the name of the court to make it sound more like a real federal court. While they were at it, they expanded the number of judgeships in order to get rid of the “liberal problem.” Killing two birds with one stone wasn’t hard with the enthusiastic support of Strom Thurmond and the armed services committees. They “promoted” the most liberal judge to a district judgeship with life tenure to get him out of town.

When the court packing began, I left and joined the staff of the newly created U.S. Court of Appeals in Atlanta. There I served for more than 20 years as the technology guru. During my tenure, we introduced personal computers, networks, VPNs, video conferencing, cellphones, automated case management systems, and the 20th century to many judges and lawyers that still were deciding cases in much the same way they had been handled since the founding of our country.

Unlike the military system, the federal courts were different. These judges weren’t wearing medals. They took their jobs seriously. All were appointed for life by the President, and most had come from cream of the crop positions in the most prestigious law firms and law schools in the country. Many that I worked for had single-handedly desegregated the South: its schools, its lunch counters, its water fountains, and even its bathrooms. No small feat considering there were still Ku Klux Klan signs announcing evening meetings when I would drive back to school on Sunday afternoons.

I had almost daily contact with these judges because our courts in the southeastern United States were handling caseloads at least 10 times what other courts across the country were experiencing. And, without the very best technology, these courts would literally have been buried by the paperwork associated with these legal proceedings. It began with civil rights cases, then voting rights, then drugs, then bankruptcies, then death penalties, and on and on.

But something changed at about the time Ronald Reagan became president. Many of the new judges that were appointed had come from prior government jobs. Some had been magistrates. Others had been state court judges. Many had been former federal or state prosecutors or lawyers in the Department of Justice. Some were small town lawyers who happened to serve as a community campaign manager for the president or a staff assistant to a U.S. Senator. There was even a former nun. What was different was that many of them came with an agenda. And many of the selections were more about ideology than legal accomplishment. We’ve now seen that process play out for more than 30 years. With some notable exceptions, many “medalists” have crept into the federal judiciary at all levels. Free resort junkets for judges sponsored by organizations with their own agenda are all too common. As we saw in the military setting, the perks of the office and living the country club life style started mattering more than doing the job. Keep in mind that these judges serve for life with almost no outside scrutiny. When you then add secret tribunals to the mix, it makes for a dangerous concoction with no checks and balances.

The 9/11 terrorists should be proud. They have single-handedly changed our entire security apparatus (not to mention our judicial system) to one that looks frighteningly similar to what you would find in Pakistan, China, or North Korea. And the response from the President and Congress… “Gosh, we’re all so much more secure when we can identify the calling history of every single cellphone user and can intercept every email communication in the universe.” And the American Technology Oligopoly appears to have gone right along with it while lying their asses off about what has happened. Who needs the keys to your house? They’ve already got access to every piece of personal data you create!

We obviously hope some of this story turns out not to be true, but then there’s this:

We share the sentiments and recommendations of Hendrik Herzberg in The New Yorker:

Calling for a national commission can be the last refuge of the high-mindedly perplexed, but this is one instance when such a commission—independent, amply funded, possessing subpoena power, and with a membership and a staff deeply versed in both national security and civil liberties—may be precisely what is needed. The N.S.A. programs represent a troubling increase in state power, even if—so far, and so far as we know—they have not occasioned a troubling increase in state wrongdoing. Obama’s “difficult questions” have a new urgency. Are the programs truly efficacious? Do they truly provide an extra margin of safety sufficient to justify the resources poured into them, to say nothing of the domestic and international anxieties they inevitably provoke? Is it wise to entrust so many of their activities to the employees of private companies, which are ultimately answerable not to the United States and its Constitution but to corporate stockholders? Did it make sense to construct an intelligence behemoth that apparently cannot operate without giving an enormous number of people—more than a million—top-secret security clearances? And in what ways, exactly, might an ill-intentioned yet formally law-abiding Administration use its powers for nefarious purposes? From what we know so far—well, we know far too little, still.

Finally, let us close by pointing you to Steve Wozniak’s commentary on this mess. We couldn’t have said it better…

Originally published: Sunday, June 9, 2013