cdr qoe rgs cdr qoe rgs help available from >

David Tucker (CEO – Event Zero) Shane Hoey (Lync MVP)

EventZero.com

Hardcore Lync Analytics

SERV303

About us

David TuckerCEO Event Zero

About us

Shane Hoey Lync MVP

We speak Australian!

http://bit.ly/KbeCIW

Hardcore Lync analyticsDefinitionLets take it to the extreme and show you what data is available from Lync and how we can use it, by using the simple tools we already use today, and some of the tools specifically designed for the purpose.

DisclaimerLync records a vast amount of information about usage. Although in most organisations this data is not private, it may contain personally identifiable data. Depending on your organisation’s “Personally Identifiable Data” policies, you may need to seek permission to view or report on the data.

Monitoring vs. Analytics ?MonitoringWe define Lync monitoring as you would think of server environmental monitoring. For example – server up/down, memory usage, disk usage and so on. There are many tools currently available for monitoring your Lync environment such as SCOM from Microsoft, Spotlight on Messaging from Quest/Dell and Microsoft Lync Performance Monitoring & Management from Solarwinds.

vs.

AnalyticsLync analytics is all about using the data produced from Lync usage such as call records and quality of experience information.

Why invest in Hardcore Analytics ?

Where do we start?

Lync ServersMonitoring Servers

Databases

Internal Clients

Edge Servers

External Clients

Monitoring Lync 2010

CDRQoERGS

SIP

Monit

ori

ng

Message

Queue

Monitoring

SIP

Monitoring

SQ

L

SQL

Monitoring

Monitoring Lync 2013

Lync Servers Databases

Internal Clients

Edge Servers

External Clients

CDRQoERGS

SIP

Monit

ori

ng

SIP

Monitoring

Important differencesLync 2010 creates interim call record updates and Lync 2013 only creates call records at the completion of a call.

Small database schema changes from 2010 to 2013.

In Lync 2013 the need for a Monitoring Server Role is removed. Monitoring Services are now collocated on Frontend Servers.

Lync 2013 does not require the Microsoft Message Bus (msmq).

Lync 2013 supports mirrored SQL databases.

Built in reports and analyticsRequires monitoring role or features, as well as Reporting Services installed on your monitoring SQL Server.

Help available from > http://technet.microsoft.com/en-us/library/gg558662.aspx

http://technet.microsoft.com/en-us/library/gg558662.aspx



Going hardcore!

Pieces of the puzzle

Setup monitoring services

Baseline

Understand thedata sources

Visualize

Setting up monitoringWhat’s required ?Enable Monitoring in TopologyAssociate with a Frontend PoolPublish Topology including Installing the DatabaseConfigure Call Detail Recording (CDR) and Quality of Experience (Qoe) IMPORTANT – Purging records from the databaseInstall SSRS and deploy monitoring reports

Anyone not setup monitoring yet ? http://channel9.msdn.com/Series/Core-Solutions-of-Microsoft-Lync-Server-2013/Core-Solutions-of-Microsoft-Lync-Server-2013-08a-DEMO-Archiving-and-MonitoringAlternatively come and see us after the session we are happy to discuss it further

http://channel9.msdn.com/Series/Core-Solutions-of-Microsoft-Lync-Server-2013/Core-Solutions-of-Microsoft-Lync-Server-2013-08a-DEMO-Archiving-and-Monitoring



Accessing monitoring and analytics dataSQL DatabasesLCSCDR for call detail recording data LCSLOG contains IM archivesQoEMETRICS for Quality of Experience data

..but wait there is more… lis location information cpsdyn call park datargcsconfig response groups configuration datargsdyn response group transient datartcdyn transient presence data rtcab address book datartcshared fabric servicertcxds backup of user dataxds Lync configuration data

Schema referenceTechnet: great source for Schema InformationCall Detail Records

http://technet.microsoft.com/en-us/library/gg398570.aspxQuality of Experience


MSDN: Lots of Lync developer focus InformationLync Developer Documentation

http://msdn.microsoft.com/en-us/library/gg455051(v=office.14).aspxMicrosoft Office Protocol Documents

http://msdn.microsoft.com/en-us/library/cc307432(v=office.12).aspx

Learners bewareHow do you explain this to your Boss?

Might be time to find a new Job!

And next time use an account with db_datareader only.

Crash course

Don’t try anything first on production servers, Test, Test, and do some more testing in a lab first.

Stick to the CDR & QoE databases as data is pushed to these.

Did we mention don’t use production servers for ideas, concepts, scripts, “but it worked on this dudes blog!” or anything similar.

Things to keep in mind

Be prepared!

Baseline your environment

Know what you’re looking for

Choose the most appropriate tools

Make an action plan!

Lets do it

Lync monitoring demo

SQL Reporting Services

Raw data to management dashboards

Number of Calls Total Time Name251 26394 minutes John, Sampat41 9310 minutes Candy, Joe45 8857 minutes Bellinger, Dodge 20 6760 minutes Busviah, Kami

133 6709 minutes Escott, Don187 5182 minutes Callens, Lieam175 4867 minutes Marquardt, Nills192 4761 minutes Vanderdas, Caroline78 4529 minutes Osorno, Fraianne

179 4397 minutes Boxman, Brian

Most active users - PowerShell

https://png.eventzero.net/enterprise-commander/lync/summary/search-voip-calls?pl=com.eventzero.plugin.lync.thirtyDaysCallsForUser&lyncUserId=47243

https://png.eventzero.net/enterprise-commander/users/summary/details?id=47243&context=users


































SQL Demo select sum(numberOfCalls) as numberOfCalls, sum(totalSeconds) / 60 as totalMinutes, UserUrifrom ( select count(1) as numberOfCalls, sum(datediff(second, InviteTime, SessionEndTime)) as totalSeconds, u.UserUri from SessionDetails s inner join Users u on s.User2Id = u.UserId where s.MediaTypes in (16, 32, 48) group by u.UserUri union select count(1) as numberOfCalls, sum(datediff(second, InviteTime, SessionEndTime)) as totalSeconds, u.UserUri from SessionDetails s inner join Users u on s.User1Id = u.UserId where s.MediaTypes in (16, 32, 48) group by u.UserUri) tbl0group by UserUriorder by totalMinutes desc, numberOfCalls desc

Call quality distribution - Excel

SQL Demoselect qualityRange, count(1) as countfrom ( select case when tbl0.quality is null then 'Unknown' when tbl0.quality = 100 then '100' when tbl0.quality >= 98 then '99-98' when tbl0.quality >= 95 then '97-95' when tbl0.quality >= 91 then '94-91' when tbl0.quality >= 86 then '86-90' when tbl0.quality >= 80 then '85-80' when tbl0.quality >= 71 then '79-71' when tbl0.quality >= 62 then '70-62' when tbl0.quality >= 51 then '61-51' when tbl0.quality >= 40 then '50-40' else '39-1' end as qualityRange, case when tbl0.quality is null then 11 when tbl0.quality = 100 then 0 when tbl0.quality >= 98 then 1 when tbl0.quality >= 95 then 2 when tbl0.quality >= 91 then 3 when tbl0.quality >= 86 then 4 when tbl0.quality >= 80 then 5 when tbl0.quality >= 71 then 6 when tbl0.quality >= 62 then 7 when tbl0.quality >= 51 then 8 when tbl0.quality >= 40 then 9 else 10 end as qualityOrder from ( select (1 - (min(a.DegradationAvg) * 0.2)) * 100 as quality from LcsCDR.dbo.SessionDetails s inner join LcsCDR.dbo.Dialogs d1 on s.SessionIdTime = d1.SessionIdTime and s.SessionIdSeq = d1.SessionIdSeq inner join QoEMetrics.dbo.Dialog d2 on d1.ExternalId = d2.DialogID inner join QoEMetrics.dbo.AudioStream a on d2.ConferenceDateTime = a.ConferenceDateTime and d2.SessionSeq = a.SessionSeq where s.SessionIdTime > '2014-01-01 00:00:00' and MediaTypes in (16, 32, 48) group by d2.DialogID ) tbl0) tbl1group by qualityRange, qualityOrderorder by qualityOrder

User geo-location – Excel

SQL Demoselect ip.IpAddress + '/' + convert(varchar(max), s.SubnetMask) as Subnet, us.UserSiteName, r.RegionName, count(distinct tbl1.EndpointId) as EndpointCountfrom QoEMetrics.dbo.EndpointSubnet esinner join QoEMetrics.dbo.IpAddress ip on es.SubnetIP = ip.IpAddressKeyinner join QoEMetrics.dbo.Subnet s on es.SubnetIP = s.SubnetIPleft outer join QoEMetrics.dbo.UserSite us on s.UserSiteKey = us.UserSiteKeyleft outer join QoEMetrics.dbo.Region r on us.RegionKey = r.RegionKeyleft outer join ( select distinct EndpointId, SubnetId from ( select s.CallerEndpoint as EndpointId, ml.CallerSubnet as SubnetId from QoEMetrics.dbo.Session s inner join QoEMetrics.dbo.MediaLine ml on s.ConferenceDateTime = ml.ConferenceDateTime and s.SessionSeq = ml.SessionSeq where s.ConferenceDateTime > '2014-01-01 00:00:00' union select s.CalleeEndpoint as EndpointId, ml.CalleeSubnet as SubnetId from QoEMetrics.dbo.Session s inner join QoEMetrics.dbo.MediaLine ml on s.ConferenceDateTime = ml.ConferenceDateTime and s.SessionSeq = ml.SessionSeq where s.ConferenceDateTime > '2014-01-01 00:00:00' ) tbl0) tbl1 on s.SubnetIP = tbl1.SubnetIdgroup by ip.IpAddress, s.SubnetMask, us.UserSiteName, r.RegionName

User communication matrix – Visualisation

SQL Demodeclare @userId intset @userId = (select UserId from LcsCDR.dbo.Users where UserUri = '[email protected]') select u1.UserUri as User1, u2.UserUri as User2from ( select User1Id, User2Id from LcsCDR.dbo.SessionDetails where User1Id in ( select @userId as UserId union select User2Id as UserId from LcsCDR.dbo.SessionDetails where User1Id = @userId and SessionIdTime > '2014-01-01 00:00:00‘ and MediaTypes in (16, 32, 48) union select User1Id as UserId from LcsCDR.dbo.SessionDetails where User2Id = @userId and SessionIdTime > '2014-01-01 00:00:00‘ and MediaTypes in (16, 32, 48) ) and User2Id is not null and SessionIdTime > '2014-01-01 00:00:00‘ and MediaTypes in (16, 32, 48) union select User2Id as User1Id, User1Id as User2Id from LcsCDR.dbo.SessionDetails where User2Id in ( select @userId as UserId union select User1Id as UserId from LcsCDR.dbo.SessionDetails where User2Id = @userId and SessionIdTime > '2014-01-01 00:00:00‘ and MediaTypes in (16, 32, 48) union select User2Id as UserId from LcsCDR.dbo.SessionDetails where User1Id = @userId and SessionIdTime > '2014-01-01 00:00:00‘ and MediaTypes in (16, 32, 48) ) and User1Id is not null and SessionIdTime > '2014-01-01 00:00:00‘ and MediaTypes in (16, 32, 48)) tbl0inner join LcsCDR.dbo.Users u1 on tbl0.User1Id = u1.UserId and u1.UriTypeId = 0inner join LcsCDR.dbo.Users u2 on tbl0.User2Id = u2.UserId and u2.UriTypeId = 0

Raw data to management dashboards3rd Party Solutions, Dossier

Microsoft SIP Processing Language(MSPL)MSPL is a scripting language for filtering and routing SIP packets.

Reference Materialhttp://msdn.microsoft.com/en-us/library/office/hh364711(v=office.14).aspx

MSPL and SDK downloadhttp://www.microsoft.com/en-AU/download/details.aspx?id=35456

MSPL is a scripting language for filtering and routing SIP packets.



SIP Message flow : Standard flow

Microsoft SIP Processing Language(MSPL)




SIP Message flow : Consume message





SIP Message flow : Replace message


SIP Proxy

SIP Proxy

SIP Proxy

SIP Proxy

SIP Proxy

CDR Rep

Lync Hardcore Analytics recapStart with the built in Reports Customize with PowerShell/SQL Extend with 3rd Party Utilities

Most Importantly be prepared for when things go wrong

Questions ?

Contact Information

David TuckerE-Mail: [email protected]: @eventzero

Shane Hoey E-Mail: [email protected]: @shanehoey

Monday, February 17th

Exhibit Hall Hours 6:00pm – 8:00pm

6:00pm – 8:00pm Welcome Reception

Tuesday, February 18th

Exhibit Hall Hours 8:00am – 9:00am (Breakfast), 10:30am – 5:00pm

8:00am – 9:00am Breakfast (Exhibit Hall) 9:00am –10:30am General Session10:30am – 5:00pm Expo Hall Hours11:00am – 12:15pm Sessions & Hands-on Labs12:15pm – 2:00pm Lunch2:00pm – 5:00pm Sessions & Hands-on Labs5:00pm – 7:00pm Ask the Experts

Wednesday, February 19th

Exhibit Hall Hours 10:30am – 4:30pm

7:30am – 8:30am Breakfast8:30am – 11:30am Sessions & Hands-on Labs10:30am – 4:30pm Expo Hall Hours11:30am – 1:00pm Lunch1:00pm – 5:45pm Sessions & Hands-on Labs6:30pm – 9:30pm Attendee Party

Thursday, February 20th

Exhibit Hall Hours 9:00am – 12:00pm

8:00am – 9:00am Breakfast9:00am – 12:00pm Expo Hall Hours9:00am – 12:15pm Sessions & Hands-on Labs12:15pm – 1:30pm Lunch and Departures

Ask the ExpertsLocation: Meal Hall located on Level 1 in Pinyon Ballroom 4-8 Tuesday, February 18

TABLE TOPICS:Best Practices, Business Value, Clients & Mobility, Lync Meetings and Video, Lync Online, Networking, Platform, Server & Manageability, Voice, Lync Feedback Sessions

Meet face-to-face with the foremost experts in the Lync field and ask them the questions that have you stumped.

Location: Breakout rooms located on Level 1 5:00pm-7:00pm

GROUPS INCLUDE:Manageability – Pinyon 2Meetings & Web Experiences – Bluethorn 4-6Mobility – Bluethorn 7-9Presence & Chat – Pinyon 1Voice & Video – Bluethorn 1-3

Come participate in targeted Feedback Sessions to hear about the high-priority feature asks and help us improve the next release!

Lync Feedback

These sessions are meant to be informational, providing an understanding of the workload and conversational, to discuss your user scenarios and desired improvements.

Birds of a FeatherBirds of a Feather flock together! Join daily breakfast discussions of relevant topics by sitting in the separately designated areas of the Meal Hall. Seating will be sorted in a different way for each Birds of a Feather breakfast:Wednesday, February 19:Where are you from? Asia/Pacific, Eastern & Central Europe, Latin America, Middle East & Africa, US (West, Central & East) and Canada, Western Europe

Thursday, February 20:What is your interest?Best Practices, Business Value, Clients & Mobility, Lync Meetings and Video, Lync Online, Networking, Platform, Server & Manageability, Voice

#LyncConf14

/msftLYNC

/microsoft-lync

/MSFTLync

Lync Launch PadYou’ve launched Lync. Now Launch this.MS Pavilion – Expo Hall

Fill out evaluations to win prizesFill out evaluations on MyLync or MyLync Mobile.Prizes awarded daily.

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

We have the information, but where to now….

Popular tools you can useData retrievalSQL Server Management StudioExcelSapian PrimalSQLPowerShellVisual StudioLync built-in reporting

VisualisationExcelPowershellVisual studioLync built-in reporting

Commercial toolsEvent Zero – Dossier tool setUnify2 – PowerviewActiance - Vantage

Other debugging tools BuiltIn ToolsPowerShellNslookupNetstatPing/PspingTelnetNetsh

Additional ToolsDigicert DDL Certificate UtilFiddlerMessage AnalyzerWiresharkPortQryNetmonTCPView