JUST RELEASED: Visit the Incredible PBX Wiki
If you enjoy calls from politicians and car warranty offers as much as we do, then today’s your lucky day. Blocking spam phone calls has been a challenge to put it charitably. Thanks to some earlier work by Stewart Nelson on the DSLR forum as well as Stewart’s considerable hand-holding in the development of our previous tutorials, we want to introduce an updated call screening approach for Incredible PBX 2021 that takes into account some recent changes in FreePBX® design. In a nutshell, the previous implementation no longer works because of some FreePBX plumbing changes that eliminated the "hook" we were using to intercept and screen incoming calls.
If you missed the earlier article, here’s the design. First time callers that are not on your WhiteList will be prompted to "press 5 to connect." Since most spam calls sit in a queue for several seconds before a live person chimes in, that person won’t hear the prompt. After 10 seconds or an invalid response, the call is sent to voicemail. In the alternative, you can play a SIT tone and disconnect the call or you can send the call to your favorite uncle, Lenny. When a successful caller calls again, the caller will be connected without a prompt. Once installed, you can change the voice prompt to a number other than 5 by modifying lines 11 and 13 of the context [sub-log-caller] which you will find in extensions_custom.conf in the /etc/asterisk directory at the completion of this install.
While today’s approach won’t block every robocaller, our testing suggests that it will catch more than 95% of these annoying calls. Using CallerID Superfecta coupled with the Asterisk® Phanebook will provide an extremely low-cost solution both for blocking spammers AND for displaying accurate CNAM data for incoming calls. The silver lining is you’ll only pay for CNAM lookups from legitimate callers once, and you have a choice of using OpenCNAM or BulkCNAM with the scripts we’ll provide today. Last, but not least, you’ll also get CNAM data for outgoing calls in your CDR logs.
Here’s the actual dialplan addition that will monitor your incoming calls:
[sub-log-caller]
exten => s,1,NoOp(*** begin sub-log-caller ***)
exten => s,n,Set(email='root')
exten => s,n,GotoIf($['${MESSAGE(from)}' != '']?SMSINCOMING)
exten => s,n,GotoIf(${DB_EXISTS(cidname/${CALLERID(num)})}?CNAMOK)
;exten => s,n,Goto(WHITELISTED)
exten => s,n,GotoIf($[${DB(SPAMCHECK/deactivate)} = 1]?CONNECTNOW)
exten => s,n,GotoIf($[${DB(cidname/0)} = SPAMCHECK]?CONNECTNOW)
exten => s,n,Playback(silence/1)
;exten => s,n,Goto(ANONTEST)
exten => s,n,Playback(custom/press5)
exten => s,n,Read(MYCODE,beep,1,n,1,10)
exten => s,n,GotoIf($["${MYCODE}" = "5"]?ANONTEST)
exten => s,n(FLUNKED),NoOp(*** Caller FLUNKED screening ***)
exten => s,n,Dial(local/*701@from-internal) ; uncomment to send to 701 VM
;exten => s,n,Dial(local/53669@from-internal) ; uncomment to send to Lenny
exten => s,n,Zapateller()
exten => s,n,Hangup
exten => s,n,Return()
exten => s,n(SMSINCOMING),NoOp(${MESSAGE(from)})
exten => s,n,NoOp(${MESSAGE(body)})
exten => s,n,GotoIf($["${email}" = "root"]?NOEMAIL)
exten => s,n,system(echo "FROM: ${MESSAGE(from)} \nVIA: ${CDR(did)} \n${MESSAGE(body)}" | `/usr/bin/which mail` -s "Incoming SMS Message" ${email})
;exten => s,n,SendText("Received but cannot reply now.")
exten => s,n(NOEMAIL),Hangup
exten => s,n,Return()
exten => s,n(CNAMOK),Set(CALLERID(name)=${DB(cidname/${CALLERID(number)})})
exten => s,n,Goto(WHITELISTED)
exten => s,n(ANONTEST),GotoIf($[${CALLERID(num)} > 0]?WHITELISTNOW:CONNECTNOW)
exten => s,n(WHITELISTNOW),Set(DB(cidname/${CALLERID(number)})=${CALLERID(name)})
exten => s,n,Set(CALLERID(all)="${CALLERID(name)} <${CALLERID(number)}>")
exten => s,n,Goto(SENDEMAIL)
exten => s,n(WHITELISTED),Set(CALLERID(all)="${CALLERID(name)} <${CALLERID(number)}>")
exten => s,n,Goto(CONNECTNOW)
exten => s,n(SENDEMAIL),NoOp(WhiteListed: ${CALLERID(all)})
exten => s,n,GotoIf($["${email}" = "root"]?CONNECTNOW)
exten => s,n,system(echo "In Asterisk Phone Book, verify new CNAM entry of ${CALLERID(name)} for ${CALLERID(number)}." | `/usr/bin/which mail` -s "Incredible PBX CNAM Reminder" ${email})
exten => s,n(CONNECTNOW),NoOp(*** end of sub-log-caller ***)
exten => s,n,Return()
;-------------------------------------------------------------------------------
The beauty of today’s design is that it won’t interfere with your existing call processing rules. In other words, FreePBX Inbound Routes sent to IVRs, Ring Groups, Conferences, and even incoming Faxes still will be processed exactly as they have been in the past once the CallerID number makes it onto your WhiteList. Unlike the previous design which tweaked the FreePBX Core module slightly, today’s implementation uses a slightly modified [app-blacklist-check] context to kick off our screening component above. This eliminates the need to modify the core module each time FreePBX updates the module. We’ve also enhanced the install to support those using the new Incredible PBX 2021 PUBLIC platform. This gives you the same SPAM protection for calls reaching your PBX directly via a SIP URI instead of a commercial DID.
Here are the basic steps to get this working:
- Configure and Enable CallerID Superfecta in FreePBX
- Enable CallerID Superfecta on All Inbound Routes
- Set the Proper Context for Your Trunks
- Download & Install Call Screener 2021
- Import Previous Callers into WhiteList
1. Configuring CallerID Superfecta in FreePBX
CallerID Superfecta is an integral component in today’s new call screening design. It will be used both to populate the Asterisk Phonebook’s WhiteList and to provide CallerID Name (CNAM) data about your callers while assuring that you only pay for one CNAM query even though grandma may call you a dozen times a day. We use the Asterisk Phonebook as the whitelist of authorized callers. The way CallerID Superfecta works is it checks multiple sources for a match on the incoming CallerID Number. As soon as a match is found, the checking ends and the CallerID Number and Name are passed to the Call Screening script.
The CallerID Superfecta lookup sequence needs to be set as follows in the United States: AsteriDex (if desired), Asterisk Phonebook (required), and then one of the following commercial CNAM lookup services: OpenCNAM or BulkCNAM. In other countries, there still may be free CNAM services, but they’ve all disappeared in the U.S. market. We’ve documented the other available sources in a previous Nerd Vittles article.
Low-volume OpenCNAM Value pricing provides global lookups for $0.0028 each. BulkCNAM provides CNAM queries with RoboCall identification for $0.002 per query. If you sign up with OpenCNAM, you will need your Account SID and Auth Token to configure CallerID Superfecta and to populate the Call Screening script. If you sign up with BulkVS, you will need your API Key from the API Credentials tab in your BulkVS Dashboard.
With your credentials in hand, login into FreePBX as admin and navigate to Admin -> CID Superfecta -> Default. Arrange and enable the lookup sources in the following order: AsteriDex, Asterisk Phonebook, and then either OpenCNAM or BulkCNAM (in the U.S. market) or your country’s best CNAM lookup source. Be sure to enter your credentials for the CNAM provider by clicking on the wrench icon beside the provider. If your incoming trunks already provide CNAM lookups (such as BulkVS and Incredible PBX Trunking), then you can substitute Trunk Provided as your CNAM lookup service. With Incredible PBX Trunking, in addition to free CNAM lookups, you also get SPAM detection at no additional cost. For details on the service, follow this link. Then we typically set Telco Data as the last lookup source which will at least give you the city and state of the caller.
2. Enabling CallerID Superfecta on Inbound Routes
By default, CallerID Superfecta is not enabled for incoming calls to your PBX. You must enable it on every Inbound Route by navigating to Connectivity -> Inbound Routes and then editing each of your routes. Then click on the Other tab and set Enable Superfecta Lookup to YES and set the Superfecta Scheme to DEFAULT. Click SUBMIT to save your route settings and then reload the dialplan when prompted.
3. Setting the Proper Context for Your Trunks
It’s equally important to make certain that the CallerID Numbers for all of your incoming calls arrive in the same format. Computers are stupid. 8005551212 and 18005551212 and +18005551212 are completely different callers as far as your PBX is concerned. If different trunks deliver calls with CallerID Numbers formatted differently, then you would need to whitelist ALL of the various permutations for every caller in the Asterisk Phonebook. For those in the U.S. and Canada that primarily receive calls from the U.S. and Canada, we recommend setting the context entry in every trunk to from-pstn-e164-us. This will handle the translation of all 3 number formats above into 10-digit numbers. Calls from other countries will not be affected.
4. Downloading & Installing Call Screener 2021
Now let’s put all the Call Screener components in place and configure the screening setup to meet your own requirements. To get started, log into Incredible PBX as root and issue the following commands:
mkdir /tmp/CALL-SCREENER cd /tmp/CALL-SCREENER wget http://incrediblepbx.com/CallScreener2021.tar.gz tar zxvf CallScreener2021.tar.gz rm -f CallScreener2021.tar.gz ./install
Once the install is begun, the editor will open to the dialplan code. In the [sub-log-caller] context, you have a few options. First, you need to choose how to handle incoming calls where the caller does not enter the "press5″ number prompt in a timely manner. The default setup (line 14) sends these callers to voicemail for extension 701. You can change the voicemail extension, or you can elect to treat the calls differently. We’ve provided two additional options. Line 15 will send the calls to Lenny at extension 53669. Line 16 will send the calls to Zapateller which is the universal tone for numbers that are not in service. You should enable only one of these three options and comment out the other two by placing a semicolon (;) at the start of the other two lines. If you have fax detection enabled on your PBX, you probably would not want to send failed calls to either Lenny or Zapateller since you may never know the incoming faxes failed. Similarly, if you get calls from people with rotary dial phones such as Grandma, you probably don’t want her talking to Lenny or listening to Zapateller tones.
The next option is which number to prompt callers to press. The default is 5. But you can change it by modifying the existing press5 entry on line 10 and entering the number to match on line 12. Available choices are 5 through 9.
The final option in the [sub-log-caller] context is to activate email notifications for new callers that pass the screening test. This is especially important if you receive lots of calls from cellphone users. Most of those calls will arrive with a CNAM entry of nothing more than the caller’s City and State. Activating an email reminder will notify you to update the Asterisk Phonebook entry for such callers to replace the City, State entry with the caller’s actual name so that your CDR listings and future calls provide accurate CNAM information for the caller. To activate email reminders, replace root in Set(email="root") in line 2 with your actual email address.
The [macro-dialout-trunk-predial-hook] context handles populating the Asterisk Phonebook WhiteList for outbound calls you make to people that are not yet in your Asterisk Phonebook. These numbers will automatically be added to your whitelist, but you also have the option of adding CNAM entries for these outbound calls using either OpenCNAM or BulkCNAM for outbound calls to numbers that are not yet in your Asterisk Phonebook. To activate CNAM lookups, simply uncomment either line 4 or 5 in the context. For the service you have activated, remember to also enter your Account SID and Auth Token in the case of OpenCNAM or your API Key in the case of BulkCNAM. If you leave both lines commented out which is the default, the callee’s phone number will be entered as both the CNAM and CNUM whitelist entry in the Asterisk Phonebook.
Once you have made all the changes desired, save the template by pressing Ctl-X, then Y, then ENTER. The installer then will complete installation of the Call Screener 2021 components.
5. Importing Previous Callers into WhiteList
We appreciate that you may not want to aggravate callers that have been calling you for years by making them jump through hoops the next time they call. So here’s a quick way to populate your Asterisk Phonebook with the names and numbers of previous callers. For entries where the CNAM is merely the CallerID Number, future calls from these numbers still will be looked up with OpenCNAM or BulkCNAM to obtain an actual CNAM match. We’ve made a couple of assumptions that you are more than welcome to adjust to meet your own needs. First, we’ve limited the list to callers from the past two calendar years. Second, we’ve only captured calls that lasted more than 15 seconds. We’ll drop down to the Linux CLI to build the list of callers to import. Then we’ll use the FreePBX GUI to import the list into the Asterisk Phonebook. While building the import list, you’ll have an opportunity to prune the list and remove any undesirable entries using nano. To generate the .csv file, issue the following commands:
cd /root ./export-CDR
Now you should have a 2YR-clean.csv file in its final form for import. Copy the file to your desktop PC and open FreePBX in your browser. Navigate to Admin -> Asterisk Phonebook. Click Import Phonebook and then Browse. Select the 2YR-clean.csv file from your desktop. Then click Upload. Take a final look at the new entries in your Asterisk Phonebook to make sure nothing came unglued, and you’re all set.
Originally published: Thursday, March 4, 2021
Need help with Asterisk? Visit the VoIP-info Forum.
Special Thanks to Our Generous Sponsors
FULL DISCLOSURE: ClearlyIP, Skyetel, Vitelity, DigitalOcean, Vultr, VoIP.ms, 3CX, Sangoma, TelecomsXchange and VitalPBX have provided financial support to Nerd Vittles and our open source projects through advertising, referral revenue, and/or merchandise. As an Amazon Associate and Best Buy Affiliate, we also earn from qualifying purchases. We’ve chosen these providers not the other way around. Our decisions are based upon their corporate reputation and the quality of their offerings and pricing. Our recommendations regarding technology are reached without regard to financial compensation except in situations in which comparable products at comparable pricing are available from multiple sources. In this limited case, we support our sponsors because our sponsors support us.
BOGO Bonaza: Enjoy state-of-the-art VoIP service with a $10 credit and half-price SIP service on up to $500 of Skyetel trunking with free number porting when you fund your Skyetel account. No limits on number of simultaneous calls. Quadruple data center redundancy. $25 monthly minimum spend required. Tutorial and sign up details are here.
The lynchpin of Incredible PBX 2020 and beyond is ClearlyIP components which bring management of FreePBX modules and SIP phone integration to a level never before available with any other Asterisk distribution. And now you can configure and reconfigure your new Incredible PBX phones from the convenience of the Incredible PBX GUI.
VitalPBX is perhaps the fastest-growing PBX offering based upon Asterisk with an installed presence in more than 100 countries worldwide. VitalPBX has generously provided a customized White Label version of Incredible PBX tailored for use with all Incredible PBX and VitalPBX custom applications. Follow this link for a free test drive!
Special Thanks to Vitelity. Vitelity is now Voyant Communications and has halted new registrations for the time being. Our special thanks to Vitelity for their unwavering financial support over many years and to the many Nerd Vittles readers who continue to enjoy the benefits of their service offerings. We will keep everyone posted on further developments.
Do you know when IncrediblePBX 2021 will be available for Raspberry Pi?
{WM: Actually, the RasPi was our early testbed for many of the features found in Incredible PBX 2021. Almost anything you can do with Incredible PBX 2021 works equally well using the existing Incredible PBX 2020 Raspberry Pi offering.]