(Trouble)shooting bad call audio

Posted on May 9, 2014 by Andrea Mocherman

Audio issues come in a smorgasbord of flavors. One way (or no way) audio. Choppy, jittery, or robotic sounding audio. Latency/delay. And last but not least, the dreaded dropped call. Whichever one you’re faced with, your users probably see it as a life-threatening plague that needs to be exterminated yesterday.

Troupleshooting-tips-for-bad-call-audio

The challenge is, in the world of digital connections, where voice packets share pathways with other network traffic, there can be any number of culprits gremlinizing your calls. Here’s how to hunt those gremlins down before they do permanent damage to your reputation as the go-to fix-it-wizard.

The Diagnosis

When audio causes trouble, instinct tells most of us to examine the connection between the PBX and the carrier. But voice connections travel in legs, starting with the initiating device. In a business setting, a typical setup has two legs before calls are passed on to a carrier; from the extension to the PBX, and the PBX out to the carrier through a demarcation point (after which your call is on the carrier’s hunting grounds). Those two legs are where you want to focus your hunt.

Your greatest weapon in audio troubleshooting is the packet capture, it’s like a blood draw you take from live calls to analyze in the lab. Once you run a capture, you’ll be able to examine each leg of the call to look for inconsistencies.

PRO TIP: We use Wireshark for all our packet captures.

The first test to run back in the lab requires headphones. Listen to the captured audio stream as it travels through each segment of the call. If you listen to the first leg, and the audio is bad, that’s where your gremlin is hiding. If the path to the PBX sings like crystal, your problem likely lies between the PBX and carrier. If that audio is clear too, check the connection at the point of demarcation. Beyond that, a polite call to your carrier explaining what you’ve found is in order – the issue likely lies with them.

Once you’ve focused your hunt on a particular segment, you can examine that interaction for trouble. So you can shoot that trouble, and maintain your hero status.

This is where your detective skills and weekend-long “Sherlock” marathon come in handy. As you examine the troubled leg, keep an eye out for these classic gremlins.

Transcoding.

If one end of the leg is working with a codec the other side doesn’t easily understand, transcoding can scramble call audio. Transcoding can lead to two other potential issues; bandwidth saturation and proprietary mismatching.

The Treatment:

Do everything you can to get all components speaking the same language. If you want to avoid any transcoding at all on your end of the calls, stick with the codecs native to the PSTN, G.711, and G.729. That way, the audio coming in from the PSTN will hit fewer twists and turns before it hits your ear, and outgoing audio will hit the network with one less disruption.

Bandwidth saturation.

If components of your system have to transcode media packets, resources are taken away from other tasks. But that’s small potatoes compared to what happens when your network is flooded with data. Large piles of data can bog down a network. If you’ve got too much data trying to stream over your network, packet flows get interrupted. It doesn’t matter if different packets from the same email show up a few more milliseconds apart, or arrive out of order, the message will still display properly. But because voice is a real-time communications medium, any significant delay in packet delivery can quickly degrade call quality.

The Treatment:

There are a couple of factors that need your attention when it comes to bandwidth saturation.

First is QoS (Quality of Service), which classifies data packets to create a hierarchy on your network. By giving voice data packets priority over the rest of the data traveling your network, you help avoid disruptions to the audio and lower the chances of voice packets getting delayed by other data that gets in their way. It’s like the difference between a four-way stop intersection and a two-way stop. You’re letting the voice data travel freely along the thoroughfare, while less time sensitive data waits until the road is clear before entering the main drag.

The more you can isolate your voice traffic from the rest of your network data, the better. You can add more segmentation by creating virtual LANs for voice, and sending voice over its own dedicated range of IPs. Like building express lanes for buses.

As far as the actual speed of your connection to the world outside your network, it really depends on how much traffic you’re sending over it and the path packets take to reach the destination. You need packets to be able to flow easily without bottlenecks. According to VoIP-info.org, one call requires approximately 64 Kbps (G.711 ) or 8 Kbps (G.729). So if you’ve got 100 simultaneous calls and a pile of data traffic pouring out over the same connection, you’ll want that connection to be fairly robust (i.e., not dial-up). But know the speeds your ISP advertises are the top speeds you might see – actual speeds will vary, and if you ask, they should be able to give you a reasonable expectation.

FitSmallBusiness has published a guide to measuring your connection so you can gauge its ability to handle your voice traffic. Remember that upload speeds are just as important as download speeds because (I assume) your phone conversations will be two sided. But that doesn’t mean you need symmetrical speeds, just hearty uploads.

If you’ve maxed out the capacity available in your region, you can always double (or triple) up your connection to dedicate bandwidth for voice. But if that’s not a budgetary possibility, there are compressors that can reduce your bandwidth consumption. Although I offer no endorsements re how adding such a utility will impact your voice quality…

Of course, you also want your connection to be pure – as direct as possible a connection to the actual Internet. If your Internet or SIP trunking provider is using a provider that’s using a provider using a provider using a provider to connect to the global network, there are a lot more legs in your connection. If that’s the case, your connection might best be represented by a late stage Jenga tower, creating more opportunity for audio problems on your calls.

Proprietary components and licensing.

If you’re using a proprietary codec, like G.729, you need to make sure you’ve paid the license fee. You’d think it would be, but it’s not necessarily included in the price of your hardware.

The Treatment:

Make sure all your licenses are up to date. Do your best, when building or updating a new system, to steer clear of proprietary that doesn’t integrate openly. Anytime you have to translate commands from one system component to another, you’re adding in the risk of transcoding. Sometimes transcoding is unavoidable, but do your best to keep everything talking the same language.

Faulty hardware.

Chances are, if your codecs are aligned and there’s bandwidth to spare across the board, hardware is the weak link.

Dropped calls are a common issue from faulty hardware. For example, something as simple as the hook (the button that gets depressed when you hang up your phone handset) could be the frequent offender. If the mechanism that performs the hangup command is defective or becomes sensitive from wear, calls can and will drop without warning.

The Treatment:

There is one fix for faulty hardware. Replace it. If it’s defective, that’s what warranties are for. If it’s worn out, well, that’s business.

The configuration caveat.

Of course some/most of the issues I’ve covered are config related. But there’s more to every telephone (VoIP) system than just a phone, a PBX, and an Internet connection. When you start introducing firewalls (like you do) and other network components/utilities, UC systems, etc, that bump shoulders (or more) with your traffic, it can get complicated.

In these cases, when you’ve run the diagnostic investigations discussed above, it’s time to start isolating the wildcards. At a time that’s convenient for everyone using the system (i.e., after regular business hours), start isolating the extra components you think could be interfering with your audio. Disconnect and reconnect them from your network one at a time, until calls sound clear. Then the fun of reintegrating the troublesome component into your system begins.

Choose your shooting partners carefully.

Because of the extreme variability found in the set up of office phone systems (not to mention networks), the ins and outs of a complete network diagnostic, and every potential error are beyond the scope of this post. Digging deeper on a complex architecture is when it becomes important to work with a voice service provider that is willing and able to work with you and your diagnostic reports (packet captures) to troubleshoot and track down a resolution no matter which side of the demarcation point the issue lies on.

With a little legwork, a detailed understanding of your network, and helpful partners, there’s not an audio gremlin you can’t troubleshoot. Good hunting.