Lync Response Group Agents Not Receiving Calls

13
3451

The Problem

After migrating from Lync 2010 to Lync 2013, a problem arose with some Response Group Agents not receiving calls. Searching the event log uncovered nothing, so my next thought was to try removing all agents from the Group, saving, and then re-adding the agents. This proved to be unfruitful, however I did notice in the event log that only the working agents were being reported as updated. When an agent is added or removed, you should see a similar event logged to the following:


Response-Group-Agent-Updated

 

So what could be the problem!? My next move was one that needs to be handled with care; I opened up SQL Management Studio, connected to the “RTC” database, and dug down to the Response Group settings – “rgsconfig” -> “Tables” -> “dbo.Agents”:

Response-Group-Agents-SQL

The first thing I noticed is the “EnterpriseVoiceEnabled” column was set to “False” for most agents. It is a requirement that agents are enabled for Enterprise Voice, so this was never going to work!

I then sanity checked that the agents were in fact enabled for “Enterprise Voice” from Lync’s perspective, they were.

Response-Group-Agent-Add

 

The Fix

As I mentioned above, please tread carefully. If you mess this up, you could have bigger problems.

  1. Right click the dbo.Agents table and select “Edit Top 200 Rows”:SQL-Edit-Top-200
  2. Edit the “EnterpriseVoiceEnable” column for each agent, replacing “False” with “True”
  3. Commit your changes using the “Update” command:SQL-Update-Command
  4. Close edit window and run the Edit or Select command again to make sure the change has stuck
  5. Check the event log for 31101 LS Response Group Service. You should see a flood of these come in after the SQL table is updated
  6. Test the call flows

 

Summary

So how did this happen? I wish I knew!! If anyone could enlighten me it would be much appreciated.

 

13 COMMENTS

  1. Great Job tracking down the issue. I wonder if you tried disabling and re-enabling the users before altering the database? This would be a more supported action to take and may have corrected the issue. Guessing the issue somehow occurred during the migration (maybe either as the agents were migrated, or the RGS themselves).

    -Kevin

      • I can confirm that removing all Agents from all Groups, then re-adding them DOES fix this, so you don’t have to dive into the database. We’re going to raise a call with Microsoft.

        Our customer had hundreds of Groups so I used this to make a CSV:

        $report = foreach($i in get-csrgsagentgroup | where-object {$_.ownerpool -like “”}) {$i.AgentsByUri | select @{n=’Response Group’;e={$i.name}},@{n=’Agent’;e={$_.LocalPath}} }
        $report | Export-Csv d:\Scripts\RGS.csv -NoTypeInformation

        then used formulas in Excel to generate a script that had this command for each line:

        $x = Get-CsRgsAgentGroup -identity service:ApplicationServer: -Name “”; $x.AgentsByUri.Remove(“sip:”); set-CsRgsAgentGroup -instance $x

        Saved that as RemoveAllAgentsFromGroups.ps1, made a copy AddAllAgentsToGroups.ps1 and replaced AgentsByUri.Remove with AgentsByUri.Add

        You’ll need to wait a couple of minutes after running the remove script for latency. The Add script will error for any Agents who have been disabled (which was an unexpected ‘tidy up’ benefit).

  2. We encountered exactly this (Agent not enabled for Enterprise Voice) during a major migration from Lync 2013 to SfB 2015 using Export/Import-CsRgsConfiguration for the RGS Migration. When we used Move-CsRgsConfiguration we did not get this problem. Did you use Export/Import or Move?

    • Hey Rick, thanks for getting in touch! I used Move-CsRgsConfiguration initially to get from Lync 2010, however I did also have to use Export/Import to move response groups between 2 Lync 2013 pools. Maybe that’s where it broke.

  3. Hi Andrew,

    Thank you for the tip, we have the exact same issue ! But each time there’s an AD/RGS synchronization (8h interval), those agents are automatically set back to EnterpriseVoiceEnabled = False on SQL.

    Did you encounter this ?

    Thanks,
    Fabien

    • You got me worried there, so I double checked. In my case the agents are still set to EV enabled. I cant think of any other reason why this might happen. Have you tried to remove users from the group manually and re-add?

      • Yes, we did remove all agents and re-add them, it didn’t resolve the issue.
        After lots of tests, the only way to make it work in our case is to disable/enable Lync users.

  4. Andrew,
    Just to report that we experienced a similar issue during a Lync 2013 to SFB migration. Removing agents and adding back in did not fix the problem. After a call with Microsoft PSS and referencing your blog post, they agreed that this was the only course of action (ie to manually edit the RTC database).

    Interestingly enough, the PSS engineer confirmed that this was not the only case Microsoft had seen. It appears that during the migration of RGS some agents have their EV attribute changed from TRUE to FALSE in the table you referenced in your blog posting.

    I hope this helps others avoid a marathon all night troubleshooting session! I will certainly be adding this to my migration procedures going forward until MS can patch the behaviour.

LEAVE A REPLY