When we first published our article suggesting that Digium® implement dogfooding for Asterisk® as a method of improving reliability, we didn’t plan to write a book. But we want you to have the full story, and information continues to trickle out. After publishing our postmortem, we just sensed that something was still missing from the picture so we followed up with a telephone call to Rod Montgomery at Digium to see if he had any further comments on the subject. To suggest that we both were a bit defensive would be an understatement, but Rod did describe Digium’s internal use of Switchvox® and briefly discussed a server running an SVN release of Asterisk. We came away from the conversation still thinking Digium was making minimal use of Asterisk in their day-to-day operation. We followed up with an email raising still more questions and presenting a draft of what we planned to publish about our conversation.
After three articles, a phone call, a half dozen emails, and three drafts of our phone conversation, Rod suddenly opened up and painted a very different picture of Asterisk use inside Digium. We want to share this final chapter with all of you. What’s both sad and disappointing is that Digium’s heart appears to have been in the right place all along. They just never told anybody.
Nobody likes self-inflicted wounds. So why did it take so long to get the whole picture? We’re still wondering, but it did prompt us to go back through all of the Digium comments to our three articles looking for something that might have revealed this sooner and avoided a lot of the animosity that you’ll find in some of the comments accompanying these articles. Many of the uglier comments weren’t published at all. But we’ll get to that.
First, the good news. Over a month ago, we recommended that Digium employ dogfooding as a method of uncovering bugs in releases such as Asterisk 1.8.4 before making them public. As it finally turns out, Digium was making extensive use of an SVN release of Asterisk to do preproduction operational testing of Asterisk. In fact, an Asterisk SVN server handles every PRI call made or received at Digium. Local TELCO trunks also connect to Asterisk and handle hundreds of calls every day. This Asterisk server also connects to a number of ITSPs for toll calling over IAX and SIP. And nearly every outbound call from Huntsville and almost all inbound calls to Digium are processed by this Asterisk server. It also accepts registrations from employees who do not regularly work out of the Huntsville or San Diego offices. Asterisk also was used for routing additional trunks during the tornado disaster that hit Huntsville a few weeks ago. And finally, this Asterisk SVN server accepts IAX and SIP URI calls from new Asterisk installs including yours. Add this to your dialplan1 or create a custom FreePBX extension if you’d like to try it for yourself:
exten => 344486,1,Dial(IAX2email@example.com/s@default)
Following Digium’s acquisition of Switchvox in 2007, a good deal of thought reportedly went into determining how best to use both Switchvox and Asterisk in a way that assured real-world usage of both systems. Digium settled on an architecture that employs Asterisk for most external-facing services and Switchvox for most internal-facing services. What this means is that desk phones at Digium actually register to a Switchvox system although incoming and outbound calls are processed by an Asterisk SVN server. That’s dogfooding in our book. While it doesn’t catch problems like the Cisco and Polycom phone issues in Asterisk 1.8.4, it exercises the software in most other respects. And we share Digium’s concern that it would be impossible to test every possible piece of hardware without assistance from the user community and the vendors.
Digium maintains (and we agree) that Asterisk needs and depends upon community collaboration because of Digium’s limited staff and the open source nature of the project. Just over 100 Digium employees simply can’t do it all. Toward that end, for those willing and able to perform some independent testing, the PIAF Dev Team has published an experimental script that installs the latest SVN checkout of Asterisk 1.8 on any existing PIAF-Purple server. We hope many of you will lend a hand! Digium also welcomes the participation of PIAF users and all Asterisk users on issues.asterisk.org.
How Do We Get There From Here? If someone says you should stop beating your wife, it’s not that helpful to your cause to suggest that your wife makes good pancakes and your neighbor sleeps with his secretary. We, of course, didn’t suggest that Digium was doing anything negative. We recommended dogfooding and better internal testing before public distribution of future Asterisk releases to improve the quality of the product. Digium may not have liked the suggestions and the recommendations may have been baseless based upon information that only Digium knew, but shooting the messenger for offering constructive suggestions and not speaking to the merits of the issues didn’t help the cause. It certainly didn’t encourage more folks to get involved in shaking out future bugs in the product.
We traced back through every comment by Digium staff in each of our articles. Unfortunately there’s nothing that could be considered a substantive response to the dogfood suggestion or to many of our readers’ comments. The practical effect was to plant the impression that there was a problem. By going negative in accusing us of a smear campaign, by not addressing the dogfood recommendation, and by telling folks to stop complaining about a product they get for free, Digium certainly shifted the discussion. But how was that helpful, especially to Digium? It’s a textbook example of what the military refers to as the Bunker Mentality, “a phenomenon that occurs when a group or individual stops taking new, pertinent information into account, and begins viewing outsiders as enemies, due to an isolation resulting from being under attack.” It’s the antithesis of “open” (as in open source) by the way.
The net result in this case was that people tended to reach their own conclusions on the merits, i.e. that something was being hidden or that dogfooding was limited to Digium’s commercial Switchvox product. That turned out not to be the case based upon facts that only Digium knew. So the negativity got in the way of a message that should have been a slam dunk and a golden opportunity to encourage more participation in the testing process specifically by folks that use Cisco and Polycom phones.2
It’s hard to unring a bell, but we want to help. We’re Asterisk’s #1 Fan! We’ve written more articles and produced more free Asterisk applications than anyone else on the planet. We appreciate everything that Digium does for the Asterisk community. So meet us half way and let’s all work to make Asterisk 1.8 an incredibly good product. It’s so close! We’ll tone down our rhetoric a bit and hopefully other fans of Asterisk will do the same. Let’s all stop thinking in terms of us against them and work toward the same objective, a better Asterisk for everyone.
For the most part, Asterisk is an open source project. As the project manager and owner of the Asterisk code, Digium bears primary responsibility for setting the tone of the discussion and nurturing a forum for open and free discussion of issues involving their project. The same goes for the Asterisk mailing lists, by the way. Sometimes you just have to grin and bear it. If Digium would prefer not to address issues that are raised on Nerd Vittles either in articles or reader comments, we’d be disappointed but that’s their call to make. They certainly have the resources to host a forum on one of their sites and participate in the discussion civilly and constructively while encouraging the entire Asterisk community to join in the conversation. Unfortunately, that didn’t happen this time around. They jumped into the discussion as an adversary but barely mentioned Digium’s internal use of Asterisk. Hopefully, we’ll all do better and be a bit more conciliatory next time around. Be gentle! Many of us are still learning. 😉
Astricon 2011. Astricon 2011 will be in the Denver area beginning Tuesday, October 25, through Thursday, October 27. We hope to see many of you there. Be sure to mention you’d like a free PIAF thumb drive. We hope to have a bunch of them to pass out to our loyal supporters. Nerd Vittles readers also can save 15% on your registration by using this coupon code. Register by July 10 to save an additional $170.
Originally published: Friday, June 24, 2011
Asterisk Security Updates. New Asterisk security releases for Asterisk 1.4, 1.6.2, and 1.8 have now been incorporated into new PBX in a Flash installs. See this link for details of the announcement.
Need help with Asterisk? Visit the PBX in a Flash Forum.
Or Try the New, Free PBX in a Flash Conference Bridge.
whos.amung.us If you’re wondering what your fellow man is reading on Nerd Vittles these days, wonder no more. Visit our new whos.amung.us statistical web site and check out what’s happening. It’s a terrific resource both for us and for you.
Awesome Vitelity Special. Vitelity has generously offered a terrific discount for Nerd Vittles readers. You now can get an almost half-price DID from our special Vitelity sign-up link. If you’re seeking the best flexibility in choosing an area code and phone number plus the lowest entry level pricing plus high quality calls, then Vitelity is the hands-down winner. Vitelity provides Tier A DID inbound service in over 3,000 rate centers throughout the US and Canada. When you use our special link to sign up, Nerd Vittles gets a few shekels down the road to support our open source development efforts while you get an incredible signup deal as well. The going rate for Vitelity’s DID service is $7.95 a month which includes up to 4,000 incoming minutes on two simultaneous channels with terminations priced at 1.45¢ per minute. Not any more! For our users, here’s a deal you can’t (and shouldn’t) refuse! Sign up now, and you can purchase a Tier A DID with unlimited incoming calls and four simultaneous channels for just $3.99 a month. To check availability of local numbers and tiers of service from Vitelity, click here. NOTE: You can only use the Nerd Vittles sign-up link to order your DIDs, or you won’t get the special pricing! Vitelity’s rate is just 1.44¢ per minute for outbound calls in the U.S. There is a $35 prepay when you sign up. This covers future usage. Any balance is refundable if you decide to discontinue service with Vitelity.
Some Recent Nerd Vittles Articles of Interest…