YATE in a Flash: Rolling Your Own SIP to Google Voice Gateway for Asterisk

A few weeks ago we introduced you to Bill Simon’s SIP to Google Voice Gateway featuring YATE. This let you set up a SIP connection to your Google Voice accounts in about 5 minutes by filling out a simple web form. Today, we take it to the next plateau for those who prefer to do it yourself. With a little assistance from Bill (about 99% of the brainpower behind what you’re about to read), we’re pleased to now offer you the alternative of creating your own SIP to Google Voice Gateway. You need not share your Google Voice credentials with anybody. Meet YATE in a Flash™.

Using today’s tutorial, we’ll show you how to create a YATE in a Flash server to which you can connect as many Asterisk® servers as you like using garden-variety SIP trunks. For those that have been using one of the last half-dozen Asterisk 10 releases in which Google Voice connectivity was totally broken and for those who have languished at Asterisk 10.0.x simply to preserve Google Voice connectivity, today’s YATE alternative is a godsend because it restores the ability to make free incoming and outgoing calls in the U.S. and Canada using any flavor of Asterisk with nothing more than a SIP trunk connection to your YATE in a Flash server. We also believe it is in everyone’s best interests to pursue other Google Voice alternatives given Digium’s recent position to no longer support Gtalk and Google Voice.

If you read Malcolm Davenport’s comment in a vacuum, you’d probably come away believing that Google Voice is just too unreliable to be a supported piece of Asterisk. Funny thing is that Google Voice still works flawlessly with Asterisk 1.8, Certified Asterisk, ObiHai devices, FreeSwitch, and, of course, YATE. We’ll let you draw your own conclusions about who is responsible for the mess with Asterisk 10. Suffice it to say, if "the community" hasn’t managed to address this in 90 days, it’s probably never going to be resolved satisfactorily… and Asterisk 11 is just around the corner. So, for once, we find ourselves in total agreement with Malcolm, "building a business based on Google Voice calling using Asterisk is not something that would be recommended." YATE appears to us to be a much more satisfactory long-term solution for those that actually rely upon Google Voice.

All of the scripts today are licensed as GPL2 code, by the way, so you’re free to embellish and enhance them to meet your own needs. Please share your improvements with us so we can pass them along to "the community."

Prerequisites. Today’s design assumes you have a server running under CentOS™ 6.2. A virtual machine works fine. While YATE runs on many other operating systems, we wanted a platform that matched our existing PBX in a Flash™ and VPN in a Flash™ environment. You will also need one or more dedicated Google Voice accounts to use in conjunction with Yate in a Flash. Do NOT use a Google Voice account with a Gmail address that you already use for email, messaging, or web phone calls!

Using the original install scripts won’t work to run YATE on an existing Asterisk server. But, if you’re a true pioneer and appreciate the risks, we’ve now included scripts for BOTH dedicated server and colocated server setups so you won’t need to make any manual adjustments. Be advised that we haven’t tested colocated YATE and Asterisk under a real-world load yet to determine what impact YATE will have on the performance of an existing Asterisk server so it’s probably not a good idea to try this on your production Asterisk machine just yet. With the low cost of virtual machine environments, there’s really no reason to run YATE and Asterisk on the same machine or virtual machine. Suffice it to say, there are many issues with conflicting port assignments for telnet, sip, and iax2 as well as listening ports. While YATE is very flexible, this colocated setup still is untested.

PBX in a Flash 2.0.6.2.5 should be on the street within the next few days or weeks. With its new all-in-one design, there will be an ISO menu option allowing you to install Yate in a Flash as a standalone server with one click. Until then, we recommend using the PIAF 2.0.6.2.4 ISO and selecting the VPN in a Flash server option. This provides an ideal platform for YATE in a Flash with the added bonus of a NeoRouter VPN server and client which happens to be the perfect way to securely interconnect your PIAF and YIAF platforms via SIP.

Overview. Yate in a Flash actually consists of several scripts. For dedicated servers (meaning Asterisk is running on a separate machine), you’ll use install-yate and add-yate-user. For colocated servers (meaning Asterisk is running on the same machine), you’ll use install-yate-on-piaf and add-piaf-yate-user. As the names imply, the first script is used to actually set up your YATE in a Flash server. The second script is used to add SIP/Google Voice accounts to the YATE server. As part of the installation process, YATE is actually compiled from source code that you’ll find in /usr/src/yate on your server. Never run install-yate more than once on the same server.

To begin, you’ll need to download and untar the YIAF tarball. Then you run install-yate or install-yate-on-piaf to get YATE installed and configured. After creating and testing your Google Voice accounts at google.com/voice, you add user accounts to YATE for each existing Google Voice account you wish to activate on your YATE in a Flash server. Each time you run add-yate-user (dedicated) or add-piaf-yate-user (colocated), the script will create a new YATE user account, Google Voice account, and SIP account on your YATE server based upon your 10-digit Google Voice number. Do yourself a favor and delete the two scripts that don’t pertain to your particular setup: dedicated or colocated. Then you won’t have to worry about using the wrong ones down the road.

Once you have YATE set up and at least one account configured, then we’ll switch to your dedicated Asterisk server and use FreePBX® to add a SIP trunk, outbound route, and inbound route for each YATE account that was created. For outbound calling, we think the easiest method to take advantage of multiple Google Voice trunks is to use a different dial prefix for each account you wish to set up.

To keep it simple, in our examples today we’ll use airport codes as prefixes so we know which Google Voice trunk is actually being used to place a call, e.g. dialing ATL-404-555-1212 (285-404-555-1212) will tell FreePBX to dial out through an Atlanta Google Voice trunk and MIA-305-555-1212 (642-305-555-1212) will tell FreePBX to dial out through a Miami Google Voice trunk. Of course, the free calls can be placed to anywhere in the U.S. and Canada regardless of the Google Voice trunk you use. However, the outbound CallerID will always be the CallerID number of the Google Voice trunk being used to place the call. Before the call is actually sent via SIP to YATE for processing via Google Voice, we’ll use FreePBX to strip off the dial prefix and add a leading 1 to match the dial string format that YATE expects to see: 1NXXNXXXXXX. If you happen to be a regex genius, this could all be done on the YATE side as well, but using FreePBX makes it easy to follow:

Installing YATE. As we mentioned, until the PIAF 2.0.6.2.5 ISO is released with the option to install YATE, we recommend you download the PIAF 2.0.6.2.4 ISO and install the VPN in a Flash server from the all-in-one menu. Once you have completed the installation of VIAF, log into your server as root and issue the following commands to install YATE:

cd /root
wget http://pbxinaflash.com/YIAF.tgz
tar zxvf YIAF.tgz

If you’re installing YATE on a separate server than your Asterisk server, then issue the following command to install YATE:

/root/install-yate

If you’re installing YATE on the same server as your Asterisk server, then issue the following command to install YATE:

/root/install-yate-on-piaf

It takes about 5 minutes for YATE to compile. Once YATE is up and running, you can monitor your YATE server using telnet. If it’s running on a dedicated server, use the command: telnet 127.0.0.1 5038. If YATE is colocated on the same server as your Asterisk machine, use this command: telnet 127.0.0.1 5039. 5038 is reserved for Asterisk. Issuing the status command will tell you what’s loaded. And we’ve found it especially handy to issue the command: debug on. This lets you track everything going on with YATE without referring to the log: /var/log/yate. To exit from your telnet session, type quit. We, of course, are barely scratching the surface of what you can do with YATE. It also can be used as a full-fledged telephony engine. Here are some examples:

Just a heads up that the version of YATE being installed comes from an svn checkout several weeks ago. We zipped it up into a tarball which is downloaded as part of install-yate. With more recent builds, we have had problems with audio and the RTP stream. Until someone can sort out the issue, you’re well advised to stick with our snapshot if you want your calls to complete successfully.

Hopefully, today’s article will bring some of the YATE gurus out of the woodwork and inspire them to share their knowledge with the rest of the VoIP community. We’d be delighted to publish further articles. It’s a truly awesome platform. As I have mentioned to some of my colleagues, it reminds me of where the Asterisk community was about seven years ago. Much of the information about YATE is buried in endless threads of mailing list messages. This is an extremely difficult way to learn about and deploy a new technology. But we’re more than willing to do our part to spread the word. We’d also be happy to add a YATE Forum to the PIAF Forums so that everyone would have a searchable collection of tips in using YATE. Let us know what you think.

Configuring Google Voice. As we mentioned, you’ll need a dedicated Google Voice account for this. The more obscure the username (with some embedded numbers), the better off you will be. This will keep folks from bombarding you with unsolicited Gtalk chat messages, and who knows what nefarious scheme will be discovered using Google messaging six months from now.

We’ve tested this extensively using an existing Gmail account, and inbound calling is just not reliable. The reason seems to be that Google always chooses Gmail chat as the inbound call destination if there are multiple registrations from the same IP address. So, be reasonable. Do it our way! Set up a dedicated Gmail and Google Voice account, and use it exclusively for this new SIP gateway. Head over to the Google Voice site and register. If you’re living on another continent, see MisterQ’s posting for some tips on getting set up.

You must choose a telephone number (aka DID) for your new account, or Google Voice calling will not work… in either direction. You also have to tie your Google Voice account to at least one working phone number as part of the initial setup process. Your cellphone number will work just fine. Don’t skip this step either. Just enter the provided 2-digit confirmation code when you tell Google to place the test call to the phone number you entered. Once the number is registered, you can disable it if you’d like in Settings, Voice Setting, Phones. But…

IMPORTANT: Be sure to enable the Google Chat option as one of your phone destinations in Settings, Voice Setting, Phones. That’s the destination we need for the SIP gateway to work its magic! Otherwise, all inbound and outbound calls will fail. If you don’t see this option, you may need to call up Gmail and enable Google Chat there first. Then go back to the Google Voice Settings.

While you’re still in Google Voice Settings, click on the Calls tab. Make sure your settings match these:

Click Save Changes once you adjust your settings. Under the Voicemail tab, plug in your email address so you get notified of new voicemails. Down the road, receipt of a Google Voice voicemail will be a big hint that something has come unglued.

Next, go into Gmail for this same account and place a test call using your new Google Voice number. You’ll find the Call Phone icon in the Chat and SMS section of Gmail in the left column. Once you complete this step, be sure to log out of both Gmail and Google Voice for this account, or inbound calling will never work.

Finally, a heads up. If you are planning to use a Google Voice account that you set up previously from a different IP address, be advised that Google has some sophisticated protection mechanisms in place to deter the bad guys. As Bill Simon discovered, this may result in your not being able to connect to Google Voice from your new YIAF server. If that happens to you, follow the steps in this Google article to unlock your account.

Adding Accounts to YATE. Now that you have your Google Voice account set up and tested, we’re ready to add an account to YATE to manage it. First, be sure you have logged out of Gmail and Google Voice for the account you plan to use, or inbound calls will never make it to YATE. You’re going to need the following information to set up a new account on your YATE server:

Google Voice account name (without @gmail.com)
Google Voice account domain (usually gmail.com)
Google Voice account password
Google Voice 10-digit phone number
YATE account name will be auto-generated
YATE account password (make it very secure!)
IP address of your YATE server (unless colocated)

If you care about security, we’d strongly recommend you consider installing a NeoRouter VPN Client on both your YATE server and Asterisk server. Use the 10.0.0.x addresses for communications between the servers, and everything will be encrypted between the machines. It also greatly simplifies the firewall and security issues. If you’ve taken our advice and installed your YATE server with VPN in a Flash, then the VPN client is already in place. Just run nrclientcmd and fill in the blanks to activate it. For tips on VPN in a Flash server setup, see this article. Be sure to write down the 10.0.0.x address of your YATE server once you get the VPN client running.

To add a new account to YATE for your new Google Voice number, log into your YATE in a Flash server as root and issue the command: /root/add-yate-user (dedicated) or /root/add-piaf-yate-user (colocated). Fill in the blanks as shown above. Be sure to write down the FreePBX Trunk settings when they are displayed. You’ll need them in the next step.

Configuring FreePBX. To finish the install, you’ll need to open the FreePBX GUI on your PBX in a Flash server using a web browser. Here are the steps. If your system doesn’t already have a default inbound route pointing to Hangup, do that first: Setup -> Inbound Routes -> Add Incoming Route.

After you have the Default Inbound Route pointing to Hangup in place, only then is it advisable to Allow Anonymous SIP Calls. Any Anonymous SIP Call not handled by an Inbound Route will immediately be disconnected. You’ll find the Allow Anonymous SIP Calls option under Setup -> General Settings or Settings -> General Settings for FreePBX 2.10:

Once you have those two pieces in place, then you’re ready to Add a new SIP trunk, Outbound Route, and Inbound Route for each new Google Voice account that you add to YATE.

1. Add SIP Trunk. Choose Connectivity -> Trunks -> Add SIP Trunk and plug in the credentials that were provided when you added your Google Voice account to YATE. We recommend numbering your SIP trunks for Yate in sequential order, e.g. YIAF1, YIAF2, etc. We’re assuming YIAF1 is your Miami Google Voice trunk in this example so ignore the 843 area code. You’re smart enough to figure out your Miami Google Voice DID for yourself. This 10-digit Google Voice DID also goes on the end of the Register String after the hash tag (/) and is not shown below:

2. Add Outbound Route. Choose Connectivity -> Outbound Routes -> Add Outbound Route. Assuming this is the Outbound Route for your Miami Google Voice trunk, fill in the form in every spot we’ve placed a pink mark like this:

These dialing rules tell PBX in a Flash to dial out through the YIAF1 SIP trunk to Google Voice whenever a user dials a 10-digit or 11-digit number with the M-I-A (642) prefix. And it tells FreePBX to strip off the 642 and add a 1 (if it is missing) before sending the call to YATE. The SIP trunk settings in YIAF1 will assure that YATE places the outbound call on the Miami Google Voice trunk when it receives 1NXXNXXXXX from Asterisk.

3. Add Inbound Route. Incoming calls from the Miami Google Voice trunk will come into Asterisk as Anonymous SIP calls with the DID of the Google Voice trunk. In order to avoid an automatic Hangup, we need to create an Inbound Route for this DID. This will be the 10-digit DID of your Google Voice trunk and will match the 10-digit number on the end of the YIAF1 trunk’s Registration String. You can route these calls in any way you like on your Asterisk system, e.g. to an Extension, a Ring Group, an IVR, or whatever. Here’s an example for you to follow. Again, please ignore the non-Miami area code. We were too lazy to fix it.

So there you have it. You’re now the proud owner of your own SIP-to-GoogleVoice Gateway courtesy of YATE and Bill Simon. You can add as many Google Voice trunks as you like. And you’ll have Google Voice connectivity with Asterisk 1.8, Asterisk 10, or Certified Asterisk without ever worrying about Asterisk "improvements" that break Google Voice down the road. To add additional trunks, do the following. On the YATE side, add-yate-user. And, on the PBX in a Flash side, complete FreePBX steps 1, 2, and 3 above using the credentials provided by add-yate-user. Enjoy!

NEWS FLASH: We are pleased to announce a new YATE Forum to provide support for YATE in a Flash as well as YATE. Come visit soon!

Originally published: Monday, June 25, 2012

Trials and Tribulations of a Service Provider. We have one of the best service providers in the business. WestNic has offered exemplary service and a secure computing platform to Nerd Vittles and PBX in a Flash for many years. We consume enormous computing resources for what we pay. But the last couple weeks have been painful. First, we were on vacation when WestNic made the transition (again) to PHP 5.3. These things usually happen in the middle of the night, and this was no exception. Unfortunately, we still were running a very old, highly customized (but very secure) version of WordPress. When morning came, Nerd Vittles died. We immediately knew why because we already had experienced PHP 5.3 a few months earlier, and WestNic graciously rolled it back… just for us. Unfortunately (for us), they didn’t tell us the new drop dead date. And, yes, we should have been updating WordPress. But it’s kinda like going to the dentist. You never quite get around to it until you have to. Well, now we had to. This involved backing up and restoring Nerd Vittles to another server still running the older version of PHP. So far, so good. It took about three hours to do the three WordPress updates, but all went well. Then we moved the site back to its home, and nothing worked again. Unfortunately, this hit on a weekend, and the weekend guys claimed it was a WordPress problem. It wasn’t this time, but it took until Monday morning to get the new php.ini file sorted out to accomodate PHP 5.3. Whew!

Then came the real fun. About 25% of the threads on the PBX in a Flash Forum could not be displayed. All you got was a blank screen when you clicked on a thread. As is customary with these types of issues, the XenForo developers blamed the provider. And the provider blamed XenForo. The provider uses mod_security to protect its web sites. But the provider assured us that nothing had changed. Well, nothing in mod_security anyway. After days and days of testing and back and forth, it turned out that the provider had added a new security mechanism, suhosin, which its developer touts as the "Guardian Angel" for PHP. That may be true for providers, but not so much for folks that actually depend upon their sites working. Welcome to a new can of worms!

Having been on both sides of this fence, we can readily appreciate the dilemma of the service providers. They don’t want their servers hacked. Denying access to all users would accomplish that goal but would reduce the number of paying customers pretty dramatically. So we all try to reach that happy medium trading off a little security for a bit more access. In this case, it turned out to be a couple of suhosin settings that monitor the length of URLs. We discovered that only after running literally hundreds of tests. Since XenForo’s forum software makes extensive use of lengthy URLs to maintain compatibility with older vBulletin posts, this caused a problem. HTML requests with URLs exceeding a certain length are simply thrown in the bit bucket by suhosin. The biggest hint was sitting in the service provider’s Apache log, but we had no access to that information, and they never looked until two and a half days after we first opened a trouble ticket. No errors appeared in our logs, and users got nothing but blank pages where the subject of a post on the forum exceeded 50 characters. Fortunately, that was enough of a hint to finally resolve the problem. The unfortunate part of this story is that, without 25 years of personal IT experience plus over 100 IT gurus that visit our sites regularly, it’s doubtful this ever would have gotten resolved other than by begging the provider to turn off mod_security and suhosin for our sites, something we were unwilling to do. If something similar ever happens to you, the command you need to know is php -v. This will tell you what’s running with PHP on your host. Our provider had implied that suhosin had not yet been activated. php -v suggested just the opposite. So did their error log once they looked. The other place to start searching for configuration information is /usr/local/lib/php.ini. This will tell you how your provider has PHP configured and whether your local php.ini file is even activated. Our provider suggested more than once that our local php.ini file had been misconfigured. We’d never touched it and, in our case, the server’s php.ini file indicated that it was never activated regardless of what its contents may have contained.

We’re glad everything is fixed. We all learned more than we ever wanted to know about suhosin. Still wishing there had been a little better communications with our provider. It would have made resolution a lot easier and quicker for all concerned. It’s especially difficult to resolve thorny issues like this using service tickets with response times of half a day per message. Did we mention there is virtually no documentation on suhosin and what each of its several dozen settings actually do. Our apologies to everyone that was impacted by the service disruptions. We’re glad it’s behind us.

Need help with Asterisk? Visit the NEW PBX in a Flash Forum.

whos.amung.us If you’re wondering what your fellow man is reading on Nerd Vittles these days, wonder no more. Visit our new whos.amung.us statistical web site and check out what’s happening. It’s a terrific resource both for us and for you.

Special Thanks to Our Generous Sponsors

FULL DISCLOSURE: ClearlyIP, Skyetel, Vitelity, DigitalOcean, Vultr, VoIP.ms, 3CX, Sangoma, TelecomsXchange and VitalPBX have provided financial support to Nerd Vittles and our open source projects through advertising, referral revenue, and/or merchandise. As an Amazon Associate and Best Buy Affiliate, we also earn from qualifying purchases. We’ve chosen these providers not the other way around. Our decisions are based upon their corporate reputation and the quality of their offerings and pricing. Our recommendations regarding technology are reached without regard to financial compensation except in situations in which comparable products at comparable pricing are available from multiple sources. In this limited case, we support our sponsors because our sponsors support us.

BOGO Bonaza: Enjoy state-of-the-art VoIP service with a $10 credit and half-price SIP service on up to $500 of Skyetel trunking with free number porting when you fund your Skyetel account. No limits on number of simultaneous calls. Quadruple data center redundancy. $25 monthly minimum spend required. Tutorial and sign up details are here.

The lynchpin of Incredible PBX 2020 and beyond is ClearlyIP components which bring management of FreePBX modules and SIP phone integration to a level never before available with any other Asterisk distribution. And now you can configure and reconfigure your new Incredible PBX phones from the convenience of the Incredible PBX GUI.

VitalPBX is perhaps the fastest-growing PBX offering based upon Asterisk with an installed presence in more than 100 countries worldwide. VitalPBX has generously provided a customized White Label version of Incredible PBX tailored for use with all Incredible PBX and VitalPBX custom applications. Follow this link for a free test drive!
 

Special Thanks to Vitelity. Vitelity is now Voyant Communications and has halted new registrations for the time being. Our special thanks to Vitelity for their unwavering financial support over many years and to the many Nerd Vittles readers who continue to enjoy the benefits of their service offerings. We will keep everyone posted on further developments.
 

Some Recent Nerd Vittles Articles of Interest…