Adding CallerID Names to VitalPBX Inbound & Outbound Calls

For VoIP users, there were no CNAM services to speak of. Thus was born our CallerID Trifecta and later CallerID Superfecta projects more than a decade ago. Superfecta was later elevated to the stratosphere by some of the early FreePBX® pioneers including Lorne Gaetz and Andrew Nagy. The real beauty of CallerID Superfecta was the ability to search a cache for a CNAM match before scouring and scraping the web, a process which took a lot more time.

Much has changed in the last 10 years. One of the early lessons in CNAM lookups obtained by scraping web sites was that most of the site owners didn’t appreciate their newfound popularity. As a result, they continually modified their sites to discourage scraping, and this in turn regularly broke CallerID Superfecta. The other major change is that there now are multiple CNAM services for VoIP users. Almost all VoIP providers also include a CNAM lookup option, and there are dirt-cheap standalone services such as OpenCNAM and BulkCNAM.

In introducing CallerID Superfecta to the VitalPBX community, we wanted to take advantage of what we’ve learned as well as the new CNAM landscape. There still are limitations. All of the CNAM lookup services don’t mind charging you to return a CNAM result of Los Angeles, CA which you could have deciphered by simply examining the area code of the incoming call. And, if your girlfriend calls you twenty times a day, the CNAM services are more than happy to charge you for the same CNAM dip twenty times. And at least one VoIP provider charges for CNAM lookups whether you want them or not. That adds another half a cent to the cost of every incoming call. Most providers charge between $.002 (new BulkCNAM rate) and $.005 per dip¹ so the charges can add up particularly with a lot of calls of short duration.

We previously mentioned using VitalPBX’s native OpenCNAM functionality to add CallerID Names to incoming calls. But there are three possible concerns with this approach. First, there’s the frequently calling girlfriend issue. Depending upon how much you like your girlfriend, that may or may not be acceptable. A second issue is the CNAM dips that only return the city and state of the call. For frequent callers, you obviously would like actual names associated with these callers. And a third issue is the absence of CNAM data in your CDR logs and on your SIP phone for outbound calls.

In considering a modern CallerID Superfecta design, we’ve concluded that you really only need three components. First, you need to examine whether an actual name was passed with the incoming call. Second, you need to check whether your local database already has a CNAM entry for the caller. And, third, if those two sources don’t bear fruit, you need a lookup from a stand-alone CNAM provider. We believe the prudent design for the local component is to consolidate all available CNAM information into a single local CNAM database with import tools from other servers for apps such as the Asterisk Phonebook, AsteriDex, and Google Contacts. The local Asterisk Phonebook included in AstDB’s SQLite3 database is perfectly suited to serve as a local CNAM repository.

Legal Disclaimer: Most CNAM providers have restrictions regarding caching of CNAM data. The courts consistently have ruled that phonebook data is not copyrightable. And every PBX caches CNAM data. After all, that’s what CDR logs are all about. Consult with your own attorney if you have concerns, or simply stop reading here. 🙂

Now that VitalPBX has included unlimited Custom Contexts at no cost, our tasks today are easy. The plan of attack goes like this. When an incoming call hits your PBX, we’ll check whether a CallerID name accompanied the call. Next, we’ll check the Asterisk Phonebook to see if we have a name associated with that phone number. If there’s no CNAM entry after checking both sources, then we’ll perform an OpenCNAM lookup using your credentials. For outgoing calls, we’ll check the Asterisk Phonebook for a CNAM match on the number being called and, if there is no match, we’ll perform an OpenCNAM lookup using your credentials. You have the option of using either, both, or neither of these CNAM scripts.

Before we get started, it’s worth noting that finding a "hook" for outbound calls was not a simple task with VitalPBX. The trick was to find a subroutine that could be replaced without damaging the native functionality of the PBX. We finally found a spot, but we were limited to just two lines of code. The solution was to use one of those lines to execute a separate subroutine which had no restrictions on size. So let’s get started.

As with FreePBX, a config file is used to manage custom contexts. With VitalPBX, extensions__80-custom.conf in /etc/asterisk/ombutel is the file you need. If it doesn’t exist, simply create it. This file has two parts which again are similar to the FreePBX design. The [cos-all-custom](+) context is used for extension code in exactly the same way that the [from-internal-custom] context is used in FreePBX. In addition to extension dialplan code, you also can embed additional contexts just as you could with FreePBX. If you’ve installed Incredible PBX for VitalPBX, then you already have a number of extension components in the [cos-all-custom](+) context. So we’ll be adding some additional contexts at the bottom of the file. If you don’t already have an extensions__80-custom.conf file, create it and add [cos-all-custom](+) as the first line in the file. All of today’s code is licensed for your use pursuant to the terms of the GPLv2 license available here.

Log into your server as root and navigate to /etc/asterisk/ombutel. Edit extensions__80-custom.conf and cut-and-paste the following code to the bottom of the file. Be sure to insert your actual OpenCNAM credentials in the SID and TOKEN entries near the top of the code replacing xxxx and yyyy.

;# // BEGIN CallerID Superfecta
[superfecta]
exten => s,1,NoOp(Executing CID Superfecta)

; insert OpenCNAM credentials below
  same => n,Set(SID=xxxx)
  same => n,Set(TOKEN=yyyy)

  same => n,NoOp(CID Number: ${CALL_SOURCE})
  same => n,Set(CID_NAME=${CALLERID(name)})
  same => n,NoOp(AstDB lookup underway)
  same => n,Set(CID_NAME2=${DB(cidname/${CALL_SOURCE})})
  same => n,GotoIf($["foo${CID_NAME2}" = "foo"]?runOpenCNAM:skipOpenCNAM)
  same => n(runOpenCNAM),NoOp(OpenCNAM lookup underway)
  same => n,Set(DB(cidname/${CALL_SOURCE})=${CID_NAME})
  same => n,Set(CID_NAME3=${SHELL(curl https://${SID}:${TOKEN}@api.opencnam.com/v2/phone/${CALL_SOURCE}?format=pbx)})
; for BulkCNAM, comment out line above and uncomment line below after inserting your APIkey
; same => n,Set(CID_NAME3=${SHELL(curl -X GET "https://cnam.bulkCNAM.com/?id=APIkey&did=${CALL_SOURCE}")})
  same => n,GotoIf($["foo${CID_NAME3}" = "foo"]?skipBOTH
  same => n,Set(DB(cidname/${CALL_SOURCE})=${CID_NAME3})
  same => n,Set(CALLERID(name)=${CID_NAME3})
  same => n,Goto(skipBOTH)
  same => n(skipOpenCNAM),Set(CALLERID(name)=${CID_NAME2})
  same => n(skipBOTH),Set(ICALL=yes)
  same => n,Answer

;  same => n,Wait(2)
;; requires addition of nv-GenericWelcome.ulaw in /var/lib/asterisk/sounds/custom
;  same => n,Background(custom/nv-GenericWelcome&silence/1&continue-in-english&press-7,nm)
;  same => n,WaitExten(.)
;; exten => 7,1,Goto(cos-all,701,1)
;; exten => 7,1,Goto(cos-all-trunk,8436060555,1)
;; exten => 7,1,Goto(newcontext,s,1)
;  exten => 7,1,Goto(incrediblepbx,s,1)
;  same => n,Return()
; exten => t,1,Hangup
;  same => n,Return()
; exten => i,1,Hangup
;  same => n,Return()

; same => n,Goto(cos-all,701,1)
; same => n,Goto(cos-all-trunk,8436060555,1)
; same => n,Goto(newcontext,s,1)
; same => n,ExecIf($["${DID}" = "8436060555"]?Goto(incrediblepbx,s,1))
  same => n,Goto(incrediblepbx,s,1)
  same => n(return),Return()
;# // END CallerID Superfecta

With the recent price drop, you may prefer to use BulkVS.com for your CNAM queries. You’ll need to obtain your APIkey from their portal in API:API Credentials. Then locate the Set(CID_NAME3… line in the script above and replace it with the following line using your actual APIkey:

same => n,Set(CID_NAME3=${SHELL(curl -X GET "https://cnam.bulkCNAM.com/?id=APIkey&did=${CALL_SOURCE}")})

We want to spend a minute on the last several lines of the Superfecta dialplan code:

; same => n,Goto(cos-all,701,1)
; same => n,Goto(cos-all-trunk,8436060555,1)
; same => n,Goto(newcontext,s,1)
  same => n,ExecIf($["${DID}" = "8436060555"]?Goto(cos-all,701,1))
  same => n,Goto(incrediblepbx,s,1)

 
These five lines demonstrate how to redirect incoming calls to five different destinations: an extension, an external phone number, another custom context, a DID-specific routing, or the Incredible PBX Demo IVR. As shown, the calls would be routed to the default Demo IVR unless the DID matches 8436060555 in which case the call would be routed to extension 701. You can modify the destination scenario simply by uncommenting a different line in the dialplan examples. Keep in mind that a Goto command permanently redirects calls to that destination outside the Custom Context so the first uncommented Goto line will always take precedence. Also note that you can have multiple ExecIf lines preceding a Goto line to route calls to different destinations depending upon the DID of the incoming call. Stated another way, multiple ExecIf commands can be followed by a Goto command that serves as the default destination for any non-matching DIDs. You could also redirect by CallerID number.

As election season heats up in the United States, be advised that your favorite legislators have exempted themselves from the robocall rules so dinnertime will get more interesting in coming months. One simple solution is to prompt the caller to enter a number before actually connecting the call. This kills most robocalls. We’ve included the code to do this in CallerID Superfecta. Just uncomment the code so that it looks like the following after choosing a call destination as documented in the previous section:

  same => n,Wait(2)
; requires addition of nv-GenericWelcome.ulaw in /var/lib/asterisk/sounds/custom
  same => n,Background(custom/nv-GenericWelcome&silence/1&continue-in-english&press-7,nm)
  same => n,WaitExten(.)
; exten => 7,1,Goto(cos-all,701,1)
; exten => 7,1,Goto(cos-all-trunk,8436060555,1)
; exten => 7,1,Goto(newcontext,s,1)
  exten => 7,1,Goto(incrediblepbx,s,1)
  same => n,Return()
 exten => t,1,Hangup
  same => n,Return()
 exten => i,1,Hangup
  same => n,Return()

 
Before this will work, you’ll need to download the nv-GenericWelcome.ulaw recording:

cd /var/lib/asterisk/sounds/custom
wget http://incrediblepbx.com/nv-GenericWelcome.ulaw

If CNAM data for outgoing calls is also desired for your CDR logs, here’s the dialplan code to cut-and-paste into the bottom of the Custom Contexts file. Insert your OpenCNAM credentials in the code replacing xxxx and yyyy just as you did in the previous section:

[cos-all-custom]
exten => _.,1,NoOp(*** Custom Outbound CNAM Code ***))
    same => n,GoSub(cnam-out,s,1)

[cnam-out]
exten => s,1,NoOp(*** Outbound CNAM context for Incredible PBX ***)
  same => n,Set(SID=xxxx)
  same => n,Set(TOKEN=yyyy)
  same => n,NoOp(Calling: ${EXTENSION})
  same => n,Set(CID_NUM=${EXTENSION})
  same => n,NoOp(AstDB lookup underway)
  same => n,Set(CID_NAME=${DB(cidname/${CID_NUM})})
  same => n,GotoIf($["foo${CID_NAME}" = "foo"]?runOpenCNAM:skipOpenCNAM)
  same => n(runOpenCNAM),NoOp(OpenCNAM lookup underway)
  same => n,Set(CID_NAME=${SHELL(curl https://${SID}:${TOKEN}@api.opencnam.com/v2/phone/${CID_NUM}?format=pbx)})
; for BulkCNAM, comment out line above and uncomment line below after inserting your APIkey
;  same => n,Set(CID_NAME=${SHELL(curl -X GET "https://cnam.bulkCNAM.com/?id=APIkey&did=${CID_NUM}")})
  same => n,Set(DB(cidname/${CID_NUM})=${CID_NAME})
  same => n(skipOpenCNAM),Set(CDR(customer_code)=${CID_NAME})
  same => n,Set(ICALL=yes)
  same => n,NoOp(*** Outbound CNAM context ends here ***)
  same => n,Return()

As with any dialplan changes, you first must reload the dialplan: asterisk -rx "dialplan reload". Next, we need to create a Custom Context within the VitalPBX GUI and point any desired Inbound Routes to that custom context. Using a web browser, log in to the VitalPBX GUI as admin. Navigate to PBX:Applications:Custom Contexts and create a new custom context for CallerID Superfecta:

Description: CID Superfecta
Context: superfecta
Extension: s
Priority: 1
Destination: Terminate Call -> Hangup

Next, identify the Trunks that you want to use with CallerID Superfecta and create an Inbound Route for those trunks pointing to CallerID Superfecta as the Destination for those calls.

If you want different destinations within CallerID Superfecta context for different trunks, then you’ll need to use ExecIf commands as shown in the dialplan to identify each trunk and to route the calls to the desired locations. Note that you can have multiple ExecIf commands followed by an uncommented default destination for any incoming calls that do not have a DID match.

To supplement your local Asterisk Phonebook which will serve as the local CNAM repository for VitalPBX, you can easily import data from other sources. And, as those sources change, you can export the entire database again and repeat the import procedure without worrying about duplicating data in your local Asterisk Phonebook. Imported data always overwrites existing entries which have a match on the phone number. To import an Asterisk Phonebook from another PBX, here are the steps to run on that PBX to create the import script which should be copied and run on the VitalPBX platform: import-asterisk-phonebook.sh

CAUTION: Carefully review the generated import script for anomalies before running it.

cd /var/lib/asterisk
sqlite3 astdb.sqlite3 "select key,value from astdb \\
 where key LIKE '%cidname/%'" > /root/import.sh
cd /root
sed -i 's|/cidname|asterisk -rx "database put cidname "|' import.sh
sed -i 's| "/| |' import.sh
sed -i 's|\\|| "|' import.sh
sed -i 's|$|"|' import.sh
sed -i "s|\"d|'d|" import.sh
sed -i "s|$|'|" import.sh
chmod +x import.sh
mv import.sh import-asterisk-phonebook.sh

Here are the steps to export existing AsteriDex data from another Incredible PBX platform. Once the steps have been completed, copy import-asteridex.sh to VitalPBX, make the script executable, and run it.

mysql -u root -ppassw0rd asteridex -N -s -e 'select "*",`out`,`name` from user1' | \\
 sed 's|\\t|, |g' > tmp.txt
sed -i "s|*,|asterisk -rx 'database put|" tmp.txt
sed -i 's|, | "|' tmp.txt
sed -i 's|$|"|' tmp.txt
sed -i "s|$|'|" tmp.txt
mv tmp.txt import-asteridex.sh

Here are the steps to import existing Google Contacts into the VitalPBX Asterisk Phonebook:

To add or replace entries in the Asterisk Phonebook from the Linux CLI, here’s the syntax:

asterisk -rx 'database put cidname 8005551212 "Directory Assistance"' 

To remove an entry from the Asterisk Phonebook, the syntax looks like this:

asterisk -rx "database del cidname 8005551212"

To display all entries in your Asterisk Phonebook, issue the following command:

asterisk -rx "database show cidname"

 
Originally published: Monday, November 4, 2019

Need help with Asterisk? Visit the VoIP-info Forum.

---

 

Special Thanks to Our Generous Sponsors

VitalPBX

1. BulkCNAM previously charged $.005 per CNAM dip but recently sent out an email indicating they were dropping the rate to $.002. Our CNAM charges continued to be billed at $.005 with the daily charges being rounded up to the whole penny. So… you may want to check your CNAM charges for a bit to verify that you are being billed at the new rate. If not, open a ticket and they will correct it. [↩]