Thanks Aaron,

I read the following article http://www.cisco.com/en/US/tech/tk652/tk653/technologies_tech_note09186a0080111b58.shtml which seems to indicate that cp tones can be generated by any device along the signal path. I am troubleshooting a very bizarre problem where even the cptones are HORRIBLY garbled. dial tone is clear, but ringing and any telco service messages are so badly distorted, that unless you just knew that someone was speaking...you wouldn't know it. If you catch my drift.

I've been on the horn with TAC and they have we have done every debug in the world and they really think its the PRI causing this horrible mess.

If you call an IP phone from the PSTN the ringback is messed up as well.

Thanks again!

Hi

I see why you're asking now.. what I said in my last post was the ideal scenario - i.e. if everything works nicely that's what should happen...

It's common to sometimes see that you don't get ringback at all when calling out, in these situations you can put a 'progress_ind alert enable 8' or something similar in which tells the gateway to tell CallManager or the originator of the call that progress tones are available 'in-band' - at which point the sound is cut through which I assume is done as an RTP stream.

If your ISDN circuit sends a similar signal or you have this configuration then it is possible that the tones are sent as audio from the PRI instead of being generate locally...

Do you have any other kit you can test on the line?

Or another line you can connect to your kit?

Aaron

Also what country are you in, and what kind of circuit do you have?

Hello Aaron,

Im in the US. The telco ran a loopback on the PRI this morning and it came up clean. This coincides with what I was seeing on my controller, no errors.

The telco is installing an analog DSL line today. I plan on piping that into one of the FXO ports and making a call. This will use the dsp resources just like on an ISDN call i have been told. If it come up clean its just another indicator that its the PRI.

On a q931 debug, i see a PI=8 and the message "in-band messages or appropriate available" which to me means that the isdn switch is sending ringback inband through an audio cut-through set up by my router.

TAC is RMAing me a new T1 module and DSPs, but they flat out told me that this would not correct the problem. Due to the audio capture we did, the TAC is convinced that the unintelligible sounds are being delivered to the router that way, by something in the signal path, the router then reproduces those sounds as is.

This is only the second customer PRI this provider has ever installed. Of course there line is spotless, it can't be there end EVER. I just want to make a phone call!!

Thanks again!

Well Cisco got me the replacement pats damn fast. I put them in and just as the TAC said it would, the problem remains.

I popped the DSL line into the fxo port and called all over the place, and it sounded just dandy.

The telco doesn't have a sunset t10 or anything like it that they could hook up to the pri and make a call on. How convenient.

The telco still says that the line is fine. But I held an induction mike up to the PRI and thought that i was listening to a Rice Krispies commercial, snap crackle and pop on the line. You could actually here pops and crackles in the line through the toner mike. But if there is THAT much noise on the line, i just can't understand why i am not seeing any conoller errors.

Any ideas?

Thanks!

An inductive amplifier type device isn't going to be able to make sense of a PRI. It's trying to replicate the 'audio' signal on the line, which on a digital carrier isn't going to sound like much more than white noise.

Go to this URL and see if you can find a sample that matches your situation:

http://www.cisco.com/en/US/tech/tk652/tk698/technologies_white_paper09186a00801545e4.shtml#stat

Based on your extensive troubleshooting with TAC, including taking what you imply to be pcm-dump captures directly out of the DSP, we can skip a lot of things.

Just for giggles, I've directed you specifically toward the 'Severe Static' (right below 'Static') sample. If it sounds just like that, you and the telco both need to go hunting around for whoever's set up for aLaw instead of muLaw. muLaw is the standard in the US; if one of you is set up for aLaw, you are set up wrong. You mentioned that your carrier is inexperienced with PRI circuits, so they might have goofed this up. The sample tells you that it will affect dial tones, but on an IP telephony system using phones like the 7912, 7940/7960, etc, dialtone is generated by the phone itself. You can verify the codecs you're using while you're on a call from the IP phone by tapping the '?' button twice in quick succession, and from the gateway using 'sh call active voice brief'.

Thanks for your input jasyoung.

A PRI through an inductive mic should sound similar to an ethernet connection. A gentle hum no matter whats going on. Once it leaves that contoller its reduced to a series of bits just like an ethernet adapter would do and sent along the wire. From a broad perspective it would appear to be the same thing. one is audio, and the oter is data, but both are bits once they leave their respective controllers.

I understand that an inductive mic wouldn't mean much on a pri. But on a totally idle PRI, with no active calls on it (because the audio is so bad), the fact that there are pops and crackles in it means that some form of unknown interference is getting injected into the path. I'm gonna get the telco to bring their mic up there and let them deny that the "noise" is there.

I made a call out of an FXO port today from a DSL line we had installed. Call quality was excellent. I would think this would eliminate the codec (which i will check tomorrow per your instructions).

The severe static doesn't even come close. Like the TAC told me, they have never heard anything like whats coming out of that PRI.

The TAC told me that when you do a pcm dump, the router make 2. I before the DSP and 1 AFTER the DSP. When he got the dump and decoded both streams, they were exactly the same he said, indicating that the data was being delivered to the router in that manner.

Thanks!

The telco showed up this morning in response to an email i had sent them. They brought their t-berd and fluke osciliscope. He hooked the t-berd inline with the pri, then out to the t1 controller in the router.

He looked at the lines and said he couldn't see anything wrong. But said he had though about this issue some last night and decided to "get smart" about the problem. He then whooped out his osciliscope and plugged it into the t-berd unit.

He had me make some test calls into and out of the system while he monitored the pcm waveform produced by their end and mine. He switched the "listening" input to my end on the t-berd and you could hear unity talking back clear as day. He switched it to listen to their input and it sounded something like the space shuttle taking off. He compared the waveforms collected by the osciliscope and said that he had never seen anything like what their isdn switch was spewing out. I saw the 2 waveforms side by side and their end was just wide band interference, mine was a true sinewave. His conclusion was that indeed their PRI is defective and they have started debugging the line. Thanks for everyone that putched in!

I made a call out of an FXO port today from a DSL line we had installed. Call quality was excellent. I would think this would eliminate the codec (which i will check tomorrow per your instructions).

You may not be the one misconfigured. As a matter of fact, you would have to go pretty far out of your way to use g711alaw instead of the default mulaw (often written ulaw or µlaw). I was more suggesting that the telco may be sending alaw audio. However, that's total random speculation on my part.

The severe static doesn't even come close. Like the TAC told me, they have never heard anything like whats coming out of that PRI.

Did TAC give you back the broken-out Rin, Sin and Sout WAV files? If so, would you mind posting them to the forum or making them available for download somewhere? A copy of 'show tech' from your router wouldn't hurt either.

The TAC told me that when you do a pcm dump, the router make 2. I before the DSP and 1 AFTER the DSP. When he got the dump and decoded both streams, they were exactly the same he said, indicating that the data was being delivered to the router in that manner.

Strictly speaking it can capture at three points:

* "Rin", the PCM form of what the IP network will be sending out.

* "Sin", the PCM data received from the PSTN on a channelized T1 or PRI, or for analog lines, the PCM output of the analog front-end chipset.

* "Sout", the PCM data after echo cancellation and other processing, but before any codec transformations.

This is intended mainly for troubleshooting the echo canceller code, but can be useful for things like this. In your situation, TAC probably only cared about Sin and Sout, and the fact that Sin was garbled meant that the audio was trashed (or alaw/mulaw mismatched) before we got it. That does tend to prove it's the PRI (or at least something in front of the DSP, the TDM bus/clocking or the VWIC). As you posted recently, the telco has just confirmed for you that audio on the PRI is messed up, so that's settled.

You might still go ahead and try the codec thing. If that's the case, your problem would be the exact opposite of the one the WAV file sample was made from, so it might sound different. You might try actually adding 'compand-type a-law' to your PRI voice-port. Again, this is rampant speculation on my part, but it would be interesting to know the result. If it works, it's the wrong solution and it needs to come back off, but it confirms exactly what's going on and what to have the telco fix.

http://www.ciscotaccc.com/voice/showcase?case=K71061213

If you have a chance, please do go ahead and post the 'show tech' and WAV files if you have them, and also any solution you arrive at with the telco. It's an interesting problem, and myself and I'm sure the other folks here are curious about the solution.

EUREKA!!!

The telco swapped out a 2 port line card in their shelf and reloaded the config to the card. They had to wait till 5;30 because their "other" pri customer was on that line card too. After about 45 min the telco called and said everything looked much more normal and they would call me back on one of the DID numbers through the system.

Everything worked and sounded great aside from a very slight echo, but i can play with the echo cancelers to corect this. After a week of bickering back and forth all it took was a swap of a line card. Sheesh, but i am glad its now working!

Thanks everyone!

Here's the wav files. This is the decoded Sin. Thats actually me speaking!

WooHoo!