CU Update Breaks SfB WebApp Audio and Video

2
1098

I recently faced an audio/video issue in the Skype for Business WebApp and I’m sharing to hopefully save someone else the pain. Updates were apparently applied to the environment a week prior, but the issue didn’t surface until about a week later. I was told that the WebApp had been tested post update, so didn’t focus too much effort on that side of things initially. Unfortunately, the updates did in fact break things due a third party call recorder that needed some settings changed to support the latest updates – Enghouse Interactive Net Service (QMS Net Service).

Here’s the story for anyone else facing this…

Overview

When joining audio/video in the meetings WebApp it just sits there forever trying to establish a connection and eventually times out. Joining the same meeting from the Skype for Business client works without issue. Summarising the issue:

  • When joining a meeting via the WebApp audio and video fails to establish, however screen sharing works
  • Same issue internally and externally
  • No problems via the Windows client

Note that while the issue I was chasing with the WebApp, this would also affect the mobile and Mac apps too because they also use the same web service.

Troubleshooting

Because the Windows client was not facing this issue I focused in on the differences in how they establish audio/video. The key difference it that the media establishment process is negotiated via the web services in the WebApp compared to native SIP signalling in the windows clients.

First, I wanted to see things from the WebApp perspective so I used Fiddler. I noticed is that the WebApp was sending its own candidates, however it never received anything back from the server:

The very next message from the server was a timeout:

Next I wanted to try and figure out why the server wasn’t sending any candidate information or whether it was getting lost in between. In this case, the issue was also seen internally with no firewalls between client and server and no internal load balancing so this ruled out these as the issue. I took a CLS log from the Front End servers and confirmed that the Front End was not sending candidates. To be sure I was looking at things correctly I compared the log to a known working deployment which clearly shows 2-way negotiation of candidates via HTTP. Looking closer it appears that in the failing case an INVITE is never sent to the conferencing server – we see initial comms from the WebApp but then comms stop and eventually the WebApp times out. You can see the differences below:

Left: Working environment – after the FE receives the initial Invite to start audio, it then invites the conf server (itself on 5062)
Right: Failing environment – initial Invite is received but there is no INVITE sent to the conf server like above. Eventually the WebApp sends a terminate request.
Still, I had no specific error. Event logs, ISS logs, WebApp logs all came out with nothing to point me to the root cause. I logged the case with Microsoft and they were also unable to find anything specific in the logs. At a loss at this stage I had a suspicion is was something UCWA related so I went back and looked at the updates that had been applied and made sure the versions where correct. I also read the update release notes and found this little gem:

“After you install the January 2019 cumulative update 6.0.9319.537 (CU8) for Microsoft Skype for Business Server 2015, the Unified Communications Web API (UCWA) applications (such as Skype for Business on Mac, a web application for UCWA, and Skype for Business mobile clients) can’t make a call or join a meeting. This issue occurs because new headers are introduced in CU8 for UCWA clients to identify the network location and third-party software that’s installed on Skype for Business Server 2015, such as Telrex and Clarity Connect. The third-party software installs its own Microsoft SIP Processing Language (MSPL) scripts which evaluate the SIP messages. These scripts cannot parse the SIP request and the request times out.”

While not specifically the issue I was facing or the CU that had been applied (in our case CU10HF1), it sparked my interest as it was UCWA related. It also states “can’t call or join a meeting” – in our case we could join the meeting but just got no audio.

Next I went looking in the Programs and Features to see what 3rd party apps were installed and I found 2:

At this point I reached out to the vendor to see if they were aware of any known issues in either of their products and I got the answer I was hoping for – yes!

Route Cause Analysis (provided by vendor)

Enghouse Interactive Net Service (QMS Net Service)

Be aware of the issues in SfB CU8+. The fix detailed below for CU8 needs to be applied to QMS customers running the QMS Net Service. There is also a fix for CU9, however this is resolved in CU10.

UCWA client cannot make a call or join a meeting after installing Skype for Business Server update 6.0.9319.537

https://support.microsoft.com/en-nz/help/4513510/ucwa-client-cannot-make-a-call-or-join-a-meeting-after-installing-cu8

Symptoms
After you install the January 2019 cumulative update 6.0.9319.537 (CU8) for Microsoft Skype for Business Server 2015, the Unified Communications Web API (UCWA) applications (such as Skype for Business on Mac, a web application for UCWA, and Skype for Business mobile clients) can’t make a call or join a meeting.

Affected users will be unable to do the following from their mobile phone on Andriod or iOS via their mobile phones

  • Make or Receive calls
  • Send or accept IM’s
  • Be unable to sign into the client

Cause
This issue occurs because new headers are introduced in CU8 for UCWA clients to identify the network location and third-party software that’s installed on Skype for Business Server 2015, such as Telrex and Clarity Connect. The third-party software installs its own Microsoft SIP Processing Language (MSPL) scripts which evaluate the SIP messages. These scripts cannot parse the SIP request and the request times out.

Resolution
Modify the QMS TelRexNetService MSPL Script, via the MSLyncComponentConfiguration.exe utility on each Front End server:

Set ‘MSPL Script’ to TelRexNetServiceNotification.am instead of the default TelRexNetService.am. This changes the mechanism which the QMS TelRexNetService service interacts with Skype SIP packets, therefore preventing the issue. You will need to restart the TelRexNetService once this is done on the front end servers.

Conclusion

If you come up against this sticky one, I hope this article gets you to a speedier resolution that me 🙂

 

 

 

2 COMMENTS

  1. Hey Andrew,

    It took me a couple of hours to find this post but after I googled “skype for business web app ucwa mspl issue” I finally got here.

    I tried rolling back the web components to version .000 which didn’t work at all, and then .534 but that would cause weird Error 500 issues with the webticket service. Seems that once you are at .559 you can’t use older web components!

    Finding this article saved me a few hours for sure. Returned to the latest update, changed the QMS settings and away it goes, happy times!

    Thanks a lot, definitely owe you a beer for this one! Keep up the good work.

LEAVE A REPLY

Please enter your comment!
Please enter your name here