Azure Information Protection Part III – AIP Scanner

The AIP Scanner is part of the AIP Client download. After you have downloaded and installed the AIP Client you can start the installation and configuration. But bevor we start the installation let’s have a look at some requirements:

• A Windows Server 2012 R2 or 2016 Server to run the service (For test and demo you can install it on a Win10 machine)
• A SQL Server 2012+ local or remote instance (Any version from Express or better is supported)
• Sysadmin role needed to install scanner service
• Service requires Log on locally right and Log on as a service right
• AIP Scanner is an AIP Premium P2/EMS E5 feature for more details review this article: https://azure.microsoft.com/en-us/pricing/details/information-protection/ 

A really good steep-by-steep description about install and configure AIP Scanner is done by Kevin McKinnerney and can be found here: https://blogs.technet.microsoft.com/kemckinn/2018/03/23/easy-configuration-of-the-azure-information-protection-scanner/

As you see in Kevins steep-by-steep guide the scanner runs as a service and uses App Authentifiction to connect with the AIP Service. So we do not need to authenticate to use the scanner.

The scanner has two main configurations which we need to configure using PowerShell:

• Add-AIPScannerRepository or Set-AIPScannerRepository -> it is about the locations and the conditions for this location
• Set-AIPScannerConfiguration -> it is about what the scanner should do during the scan

Add-AIPScannerRepository

This cmdlet adds a so called data repository to be scanned and creates a profile of settings. For example, you can specify a default label for unlabeled files, and whether to override an existing label or not. We can specify local folders, UNC paths, and SharePoint Server URLs for SharePoint sites and libraries. The scanner can handle more than one data repository. So you can configure a mix of local folders, UNC paths and SharePoint Server URLs with different setting covered by one AIP Scanner installation.

To change this settings we can use: Set-AIPScannerRepository cmdlet. To remove a data repository use: Remove-AIPScannerRepository cmdlet.

Example:

Set-AIPScannerRepository -Path C:\Temp2 -SetDefaultLabel UsePolicyDefault -MatchPolicy On

To review the settings, we can use Get-AIPScannerRepository. As you can see in my example I have configured two repositories with different settings:

To get more examples and details review the official documentation: https://docs.microsoft.com/en-us/powershell/module/azureinformationprotection/add-aipscannerrepository?view=azureipps

Set-AIPScannerConfiguration

Set-AIPScannerConfiguration cmdlet is used to configure settings for the AIP Scanner. These settings include:

• Discovery mode or applies labels
• File will be relabeled YES or NO
• File attributes are changed YES or NO
• What is logged in the reports
• Scanner runs once or continuously
• Justification message used when required
• Rights Management owner for protected files

Example:

Set-AIPScannerConfiguration -Enforce On -Schedule Manual -DiscoverInformationTypes All

To get more examples and details review the official documentation: https://docs.microsoft.com/en-us/powershell/module/azureinformationprotection/set-aipscannerconfiguration?view=azureipps

Set-AIPScannerScannedFileTypes

This cmdlet is used to let the scanner know which files types should be scanned.

The cmdlet sets a list of file types to scan or exclude from scanning. To scan all file types, use *. To scan only specific file types use *.<file name extension>. To exclude specific file types from being scanned use -*.<file name extension>. And to reset the list back to default use @().

If no data repository is specified the setup applies to all data repositories that do not have their own list specified.

To get more examples and details review the official documentation: https://docs.microsoft.com/en-us/powershell/module/azureinformationprotection/set-aipscannerscannedfiletypes?view=azureipps

Scenarios

The scanner can typically be used for the following scenarios. Reports are stored in this location: %localappdata%\Microsoft\MSIP\Scanner\Reports

Scan for sensitive information types

#Configure data repository:

Add-AIPScannerRepository -Path C:\Temp2

#Configure Scan: Scan for all known sensitive types

Set-AIPScannerConfiguration -Enforce Off -Schedule Manual -Type Full -DiscoverInformationTypes All

#Start Scan

Start-Service AIPScanner
Start-AIPScan -reset

Label / Protect files

#Configure data repository:

Add-AIPScannerRepository -Path C:\Temp2 -OverrideLabel On -DefaultLabelId ae7eaeb0-cfdf-4217-a895-32a6b41311d9 -MatchPolicy Off

#Configure Scan: Scan for all knowen sensitive types

Set-AIPScannerConfiguration -Enforce On -Schedule Manual -ReportLevel Debug -Type Full

#Start Scan

Start-Service AIPScanner

Start-AIPScan -reset

Scan for sensitive information types and labels and protect files that match

#Configure data repository:

Add-AIPScannerRepository -Path C:\Temp2 -OverrideLabel On -MatchPolicy On

#Configure Scan: Scan for all knowen sensitive types

Set-AIPScannerConfiguration -Enforce On -Schedule Manual -Type Full -DiscoverInformationTypes All

#Start Scan

Start-Service AIPScanner

Start-AIPScan -reset

Related posts:

• Azure Information Protection Part I – Overview
• Azure Information Protection Part II – PowerShell
• AzureInformation Protection Part IV – Work with AIP
• Azure Information Protection Part V – Advancedfeatures

Azure Information Protection Part II – PowerShell

This article is on overview about the functions and scenarios using PowerShell in the context of Azure Information Protection. Everything in this article is based on the official Microsoft documentation.

Microsoft published a brilliant Admin Guide about using PowerShell with Azure Information Protection containing all details and scenarios: Admin Guide: Using PowerShell with the Azure Information Protection client.

Overview

In Azure Information Protection we can use PowerShell to:

• Administering Azure Information Protection
• Configuration for the super user feature
• Using Azure Information Protection
• Work with the AIP Scanner

Azure Information Protection knows two PowerShell modules.

• AADRM: These cmdlets are used to administer the protection service (Azure Rights Management) for Azure Information Protection.
• AzureInformationProtection: These cmdlets are used to protect files, label files, and get information about files.

First step is to install the AADRM PowerShell module. To do this open PowerShell and use: Install-Module -Name AADRM. For more details and requirements about how to install AADRM PowerShell modules read this article: Installing the AADRM PowerShell module

To get an overview of all cmdlets use: Get-Command -Module AADRM or look at this list: Cmdlets grouped by administration task

The AzureInformationProtection cmdlets are part of the Azure Information Protection client. These cmdlets can be used with Azure Information Protection, the protection service (Azure Rights Management), and Active Directory Rights Management Services (AD RMS).

To get an overview of all cmdlets use: Get-Command -Module AzureInformationProtection

Main scenarios and some examples

We can use PowerShell for AIP for classification and protection. Let’s have a deeper look on some typical scenarios:

Scenario 1: remove protection from files for others using your own account

In this scenario we need PowerShell to configure the “super user feature”. In addition, we need to configure your account to be a super user for Azure Rights Management. To enable the super user feature, use the PowerShell cmdlet Enable-AadrmSuperUserFeature To assign your user or a groups we can use the Add-AadrmSuperUser cmdlet.

To get a complete overview about the super user feature in AIP refer to this official article: Configuring super users for Azure Rights Management and discovery services or data recovery

The super user feature is also needed if we need to index mailboxes for search operations or if DLP solutions. CEG or anti-malware products can also use the super user to inspect files that are protected by AIP.

Scenario 2: protect or unprotect files without user interaction

First step in this scenario is to connect to Azure Rights Management Service by using this cmdlet: Connect-AadrmService. To login enter your Azure Information Protection tenant administrator credentials.

To get an overview about the ARM instants and to verify that the login was successful use: Get-AadrmConfiguration. The result will be something like this for example:

In the next step we will use PowerShell to get a list of all available Labels using this cmdlet: Get-RMSTemplate. In my example it looks like this:

Now we can to several things:

• Protect a file: Protect-RMSFile -File C:\Test.docx -InPlace -TemplateId e6ee3481-26b9-45g5-b33a-f774escd43b0
• Protect all files in a folder: Protect-RMSFile -Folder \ServerABC\Docs -InPlace -TemplateId e6ee3481-26b9-45g5-b33a-f774escd43b0
• Get the status of a file: Get-RMSFileStatus -File \Server1\Documents\TestABC.docx
• Remove protection: Unprotect-RMSFile C:\testDoc1.docx -InPlace

Related posts:

• Azure Information Protection Part I – Overview
• Azure Information Protection Part III – AIP Scanner
• AzureInformation Protection Part IV – Work with AIP
• Azure Information Protection Part V – Advanced features

Using site classification for SharePoint Sites

Site classification is a must-have when we talk about Governance, Compliance and also topics around GDPR.

Beside 3^rd party solutions focusing on site and content classification we have also some out of the box options and developer opportunities in Office 365 and SharePoint on-prem. Depending on if we are talking about classic SharePoint Site Collections or if we talk about modem Team Sites, being part of an Office 365 Group, we have different szenarios.

To create a new SharePoint site in Office 365 we know two different ways.

1. We can create a SharePoint Online Site using the SharePoint Online Administration. This will create a SharePoint Site based on WebTemplate STS
2. We can go to SharePoint Home and click “create” in the upper left corner or we can go to Outlook Online and create a new Group. Both will create a SharePoint Site based on WebTemplate GROUP

To provide a site classification solution for classic Team Sites created by option 1 we need to implement the following: Implement a SharePoint site classification solution. This works also for SharePoint 2013 on-prem. The article describes a full solution including policies for site closing and deletion depending on the classification setting. As you can see the article describes some steps to do:

• Define and set site policies
• Insert a custom action
• Custom site classification
• Add a classification indicator to site page

Using the opportunities we have with Groups and Group Policies some of these things can be automatically put to a SharePoint Site based on WebTemplate GROUP.

This video by Vesa Juvonen is showing the steps and the final results:

As you can see we need to create the site bases on option 2.

(Dialogs already including policies)

SharePoint Home - Create:

Outlook Online -> Create Group:

Final result:

Step by Step

To enable this functionality in Office 365 we need to set up an “Settings Object” and a “Settings Template” in Azure AD. To do this we can use the Azure Active Directory cmdlets for configuring group settings.

First of all we need to install the preview of Azure Active Directory V2 PowerShell Module:

Install-Module -Name AzureADPreview

To set up the site classification options and configure properties like ClassificationList and ClassificationDescriptions etc follow these steps also shown in Vesas video:

#Connect

Connect-AzureAD

Get-AzureADDirectorySettingTemplate

#Create

$Template = Get-AzureADDirectorySettingTemplate -Id 62375ab9-6b52-47ed-826b-58e47e0e304b

$Setting = $template.CreateDirectorySetting()

$setting["UsageGuidelinesUrl"] = "http://sharepointtalk.com"

$setting["ClassificationList"] = "Public, Internal, TopSecret"

$setting["DefaultClassification"] = "TopSecret"

$setting["ClassificationDescriptions"] = "Public:no restrictions,Internal:all internal users can access,TopSecret:only special users can access"

$setting["GuestUsageGuidelinesUrl"] = "http://sharepointtalk.net"

New-AzureADDirectorySetting -DirectorySetting $setting

#Check

Get-AzureADDirectorySetting -All $True

(Get-AzureADDirectorySetting -Id %%YOUR ID%%).values

As described in the video we can now use the CLASSIFICATION property to assign a site policy or any other custom action. Details about site policies are part of Implement a SharePoint site classification solution.

Here the script taken from the video to get the CLASSIFICATION property:

#Get PnP PowerShellOnline

Install-Module SharePointPnPPowerShellOnline

#Get Site classfication value

Connect-PnPOnline https://%YOUR TENANT%.sharepoint.com/sites/%YOUR SITE%

Get-PnPSite

$Site.Classification

Get-PnPProperty -ClientObject $Site -Property Classification

Overview of shared with Externals and shared Anonymous in Office 365

The GDPR highlights the need for protection of personal data held by organizations. To be able to do this Microsoft inverted a lot in new features and functions like the Office 365 Security & Compliance Center or the GDPR Assessment.

One of the backend systems helping to fulfill those regulations is the SharePoint Online Search Service. In the SharePoint Online Search schema, we can find two managed properties focusing on sharing and access from outside of your organization.

ViewableByExternalUsers and ViewableByAnonymousUsers

Both had the same setting: Query, Retrieve, Refine and Sort. So we can use them to create some reports based on search queries.

Personal overview

Office 365 let every user search in his SharePoint Online sites, OneDrive for Business files and also in Emails for content. In this scenario Email is of topic. But using this search function at the landing page of Office 365 a user can create a personal overview of content he shared to externals or anonymous.

To do this a user needs to fill in the following query in the search box at the Office 365 landing page:

ViewableByAnonymousUsers=true

In this example, I search for documents located in SharePoint Online sites or in my personal OneDrive for Business which are shared based on an anonymous guest link.

Using the query ViewableByExternalUsers=true shows me the files shared with external users through a sharing link that requires them to log in before they can view the file.

This gives a user an overview of documents he has shared from his OneDrive for Business with externals or anonymous. Because the URL is generic you can use this link for all your users and every user get his person overview: https://www.office.com/search?auth=2&home=1&q=ViewableByAnonymousUsers%3Dtrue

Also you can use this link to create a tile in the Office 365 App Launcher as described in the article: Add custom tiles to the app launcher

The result may look like this:

Team Site overview

Microsoft integrated a new out the box reporting capability in every Team Site. The article: View usage data for your SharePoint Online site is showing all details you need to know. There is also a new tab called “Shared externally”.

The article says: List of files you have access to that have been shared with users outside your organization through a sharing link that requires them to log in before they can view the file. Files shared with anonymous users or files available to users with guest permissions are not included.

To get a list of files shared anonymous in this Team Site we can again use the query: ViewableByAnonymousUsers=true followed by a path filter like for example: path:https:\\yourTeamSiteName.sharepoint.com.

Using Search Center to get an overview

As an administrator, you can also use the search center to get an overview of anonymous shared content or about data and also SharePoint Online Sites them self, shared to externals. The queries are basically the same and you can extend them with additional keyword queries properties.

For example, search all Office 366 Groups external users can access:

ViewableByExternalUsers=true contentclass:sts_site WebTemplate:GROUP

(Because of security trimming in SharePoint Search the user who runs the query needs access to all Team Sites to gets an complete report.)

Of cause there are also options archiving this using PowerShell for Office 365 Groups or using Reports in the Office 365 Security & Compliance Center. Using the SharePoint Online search gives you the power and flexibility to integrate all managed properties as metadata in you report like for example ViewsLifeTime, LastModifiedTime, CreatedBy or ModifiedBy. In addition you can easily scope your report to only show documents using the IsDocument=true query parameter or to focus to special Site Templates like WebTemplate:GROUP to only show Office 365 Groups Team Sites etc.

Using PowerShell to get the report

Using PowerShell to get results from SharePoint Online Search also offers the option to save the report as an *.csv file. To call SharePoint Online Search API using PowerShell and save the result to an *.csv file you can follow the steps explained by Prasham Sabadra in his article Office 365/Sharepoint Online - PowerShell Script To Call Search API And Get The Result.

This example is based on his description. The report is showing all external shared content and sites in an Office 365 Tenant and is saving the result to C:\Temp\ViewableByExternalUsers.csv

# add references to SharePoint client assemblies and authenticate to Office 365 site - required for CSOM   

Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"   

Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"   

Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Search.dll"

#Specify tenant admin and URL 

$User = "Admin@yourTenant.onmicrosoft.com"   

#Configure Site URL and User 

$SiteURL = "https://yourTenant.sharepoint.com"  

#Password 

$Password ="yourPassword"   

$securePassword = ConvertTo-SecureString -String $Password -AsPlainText –Force  

$Creds = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($User,$securePassword)

#client context object and setting the credentials  

$Context = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL) 

$Context.Credentials = $Creds

#Calling Search API - Create the instance of KeywordQuery and set the properties 

$keywordQuery = New-Object Microsoft.SharePoint.Client.Search.Query.KeywordQuery($Context)  

#Sample Query - To get the last year result 

$queryText="ViewableByExternalUsers=true" 

$keywordQuery.QueryText = $queryText 

$keywordQuery.TrimDuplicates=$false 

$keywordQuery.SelectProperties.Add("LastModifiedTime") 

$keywordQuery.SelectProperties.Add("ViewsLifeTime") 

$keywordQuery.SelectProperties.Add("ModifiedBy") 

$keywordQuery.SelectProperties.Add("ViewsLifeTimeUniqueUsers") 

$keywordQuery.SelectProperties.Add("Created") 

$keywordQuery.SelectProperties.Add("CreatedBy") 

$keywordQuery.SortList.Add("ViewsLifeTime","Asc")

#Search API - Create the instance of SearchExecutor and get the result 

$searchExecutor = New-Object Microsoft.SharePoint.Client.Search.Query.SearchExecutor($Context) 

$results = $searchExecutor.ExecuteQuery($keywordQuery) 

$Context.ExecuteQuery() 

#Result Count 

Write-Host $results.Value[0].ResultRows.Count

#CSV file location, to store the result 

$exportlocation = "C:\Temp\ViewableByExternalUsers.csv" 

foreach($result in $results.Value[0].ResultRows) 

{ 

$outputline='"'+$result["Title"]+'"'+","+'"'+$result["Path"]+'"'+","+$result["ViewsLifeTime"]+","+$result["ViewsLifeTimeUniqueUsers"]+","+$result["CreatedBy"]+","+$result["Created"]+","+$result["ModifiedBy"]+","+$result["LastModifiedTime"] 

Add-Content $exportlocation $outputline  

}

Deactivate social feature MySite, Newsfeed, OneDrive, Sync, Share & Follow in SharePoint 2013

In several project I had the request to deactivate one or all of the following features. Poorly there is no consistent way to do it.

Sync button

To deactivate that feature we can use the SPList.ExcludeFromOfflineClient property. With this PowerShell Script we can set the value to 1 which deactivates the Synch function everywhere.

THX Sahil Malik for that script

Get-SPSite -limit all | get-SPWeb -limit all | Foreach {$_.Title = $_.Title;

$_.ExcludeFromOfflineClient=1; $_.Update()}

MySite:

If we can start with a fresh system we can remove the permissions to create MySites. If already MySites are created we first must delete them. Only the MySites, not the MySite host!

http://technet.microsoft.com/en-us/library/ee721063(v=office.15).aspx

Do not give the user the option to set up MySites will also remove the Newsfeed and the OneDrive link in the SharePoint top link bar.

Share

This script disables the “Allow access request” setting under Site Settings -> Site Permissions -> Allow access request for all Sites and SubSites und the configured URL

$Webapp = Get-SPWebApplication "http://MySharePointURL"

ForEach ($site in $webapp.Sites)

{ForEach ($web in $site.AllWebs | where {$_.RequestAccessEnabled -and $_.Permissions.Inherited -eq $false})

{$web.RequestAccessEmail=""

write-host $web.Title, $web.URL updated}}

The “Share” button and also the entries in the HoverPannels and context menus are still there but the user get the message that he did not have the right to share content or sites. Another option is to set the max allow documents, sites and users to follow to 0. We can do this in Manage Profile Service -> MySites settings section -> Manage Following http://technet.microsoft.com/en-us/library/jj219740(v=office.15).aspx

Follow

This script goes through the hierarchy and disables the Following Content feature in every WebSite. You get an error message for every site where the feature is not active or missing. This can be ignored

$w = Get-SPWeb http://MySharePointURL | ForEach{ $_.URL }

Get-SPFeature -Web $w |%{ Disable-SPFeature -Identity "FollowingContent" -URL $w -Confirm:$false}

Query SharePoint Search Analytics using PowerShell

Analytics have been completely revised in SharePoint 2013. Some of the core tech in the previous FAST release has been fused together with SharePoint enterprise search to create a new and powerful analytics engine.

The Analytics Engine is based on two main parts. We have Search Analytics and we have Usage Analytics features. Both parts have own reports which can be found under Site setting and as part of the Search Service Application in Central Admin.

Usage Analytics out-of-the-box report:

Search Analytics out-of-the-box report:

By using PowerShell we can query both Analytics parts and get custom reports.

Usage Analytics:

First of all we have to set the Search Service Application:

 $searchApp = Get-SPEnterpriseSearchServiceApplication

Next step is setting up a site scope for the report:

$Site = Get-SPSite http://intranet.contoso.com/sites/contoso/

Now we can execute the request:

$result = $searchApp.GetRollupAnalyticsItemData(1,[System.Guid]::Empty,$Site.ID,[System.Guid]::Empty)

If the report should not have a site focus we had to replace the “$Site.ID” part with “[System.Guid]::Empty”.

To focus the report data to a specific day or month we can use the GetHitCountForDay and the GetHitCountForMonth method.

First we had set up the data to which we want to focus:

$FilterDate = Get-Date "2014-01-20"

Then we can use the $FilterDate variable together with both methods:

$result.GetHitCountForDay($FilterDate)

$result.GetHitCountForMonth($FilterDate)

Search Reports:

Search Analytics cmdlets follows a different syntax. To get data from that engine we also had to setup the 

Search Service Application:

$searchApp = Get-SPEnterpriseSearchServiceApplication

Also in this case we can focus to a specific site using a variable:

$site = Get-SPSite http://intranet.contoso.com/sites/contoso/

The difference now is that we had to set up a date time value for the report. Leaving it blank is no option here:

$searchApp.GetSearchReport(1,[System.Guid]::Empty,[System.Guid]::Empty,[System.DateTime]::Now,$false,100)

That command gets the report data for the actual month: [System.DateTime]::Now . 

The next parameter which is set to “false” is the switch for getting data for the month = false or the day = true.

If we want the data for another month than the we have to set up a date variable and use it instead of [System.DateTime]::Now :

$FilterDate = Get-Date "2014-01-20"

$searchApp.GetSearchReport(1,[System.Guid]::Empty,$site.ID,$FilterDate,$false,100)

Getting farm wide data we also had to replace the $site.ID variable with “[System.Guid]::Empty”:

$searchApp.GetSearchReport(1,[System.Guid]::Empty,[System.Guid]::Empty,$FilterDate,$false,100)

The first parameter with the value “1” is setting the Event Type. Event Type 1 = views. The next parameter is the TenantID, followed by SiteID, date value, month / day switch and the max result value (100 means show me the top 100 search terms).

Using SharePoint PropertyBag in the Context of Search

My first post @ MVP Award Blog :-)

http://blogs.msdn.com/b/mvpawardprogram/archive/2013/11/25/using-sharepoint-propertybag-in-the-context-of-search.aspx

again with a hands on lab video @ my YouTube channel:

 

 

 

PowerShell for SharePoint 2013 Analytics Feature

First of all
Thanks a lot to Thomas Molbach from Microsoft. He sends me his Scripts from his Session SPC 202 from last year SharePoint Conference in Las Vegas. So most of the PowerShell commands in this article are based on his scripts.

Working with PowerShell and the new SharePoint Analytics feature means that you have to set up your PowerShell environment first. Using the SharePoint Management Shell as it is won’t work.

Common stuff
To setup up a Management Shell you need to run the following script:

#Set up the Anaytics powershell environment

Add-PSSnapin Microsoft.SharePoint.PowerShell

Add-PSSnapin hostcontrollerpssnapin

Add-pssnapin junopssnapin

Add-pssnapin searchcorepssnapin

Add-pssnapin enginepssnapin

Add-pssnapin analysisenginepssnapin

$env:CERES_REGISTRY_PRODUCT_NAME = "Office Server\15.0\Search\Ceres"

Connect-System -Uri (Get-SPEnterpriseSearchServiceApplication).SystemManagerLocations[0] -ServiceIdentity (Get-SPEnterpriseSearchService).ProcessIdentity

Connect-AnalysisEngine -NodeName AdminComponent1

The major part is: Add-pssnapin analysisenginepssnapin . As you see the Analytics feature is not part of the common SharePoint PowerShell Snapin.

If everything worked fine you get the following status back:

Connected to agent 'AnalysisEngineComponent' of type Microsoft.Ceres.AnalysisEngine.Shared.IAnalysisEngineAgent running

Now we are able to get a full list of available command using:

Get-Command -Module analysisenginepssnapin

Some useful scripts

 

Start Usage Analytics Job:

Start-Analysis UsageAnalytics

 

Start Search Reports Job:

Start-Analysis SearchReports

 

Start Search Analytics Job:

$tj= Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition

$sa = $tj.GetAnalysis("Microsoft.Office.Server.Search.Analytics.SearchAnalyticsJob")

$sa.StartAnalysis()

 

Display Search Analytics Job status info:

$a = Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition

$sa = $a.GetAnalysis("Microsoft.Office.Server.Search.Analytics.SearchAnalyticsJob")

$sa.GetAnalysisInfo()

Display all Analytics Job status info:

Get-Analysis

SharePoint 2013 Analytics Feature - IT-Pro, DB Admin & PowerPivot issues

This post covers some facts and details about the new SharePoint 2013 Analytics feature. For common stuff about that feature have a look at this TechNet article: LINK

In fact we have two different types of analytics in SharePoint 2013:

·         Search analytics analyzes content that is being crawled and added to the search index

·         Usage analytics analyzes user actions, or usage events, such as clicks or viewed items, on the SharePoint site

Search Analytics & Usage Analytics processing is based on several components. Continuous or incremental crawls must be scheduled and run. Also there are several timerjobs involved:

·         Analytics Timer Job for Search Service Application Search Service Application

·         Microsoft SharePoint Foundation Usage Data Import

·         Microsoft SharePoint Foundation Usage Data Processing

·         Usage Analytics Timer Job for Search Application Search Service Application

For more details about the jobs and job descriptions have a look here: LINK

This picture shows an overview for the new components:

Search analytics

PowerShell

Some Facts:

·         There are some cmdlets for create / delete/ get / move etc.  SharePoint Analytics infrastructure issues:

o   SPEnterpriseSearchAnalyticsProcessingComponent

o   SPEnterpriseSearchLinksDatabase

·         There are no direct PowerShell cmdlets to get informations from the SharePoint Analytics

·         We can use the .dot net SearchServiceApplication class, from Microsoft.Office.Server.Search.dll. in combination with PowerShell to work with SharePoint Analytics

Here is an example script from Radu Tut showing how we can connect via PowerShell to SharePoint Analytics:

$searchApp = Get-SPEnterpriseSearchServiceApplication

$site = Get-SPSite "{SiteUrl}"

$result = $searchApp.GetRollupAnalyticsItemData(1,[System.Guid]::Empty,$site.ID,[System.Guid]::Empty)$result

#for a specific date or for a specific month

$date = Get-Date "2013-01-18"

$result.GetHitCountForDay($date)

$result.GetHitCountForMonth($date)

So for example using the first part of the script:

$searchApp = Get-SPEnterpriseSearchServiceApplication

$site = Get-SPSite "{SiteUrl}"

$result = $searchApp.GetRollupAnalyticsItemData(1,[System.Guid]::Empty,$site.ID,[System.Guid]::Empty)

$result

Gets the following result in my demo environment:

There are two methods available. You can use:

·         GetSearchReport

·         GetRollupAnalyticsItemData

Find complete post and info’s about using this in Radu Tuts blog here:  How to get Search Analytics Reports programmatically in SharePoint 2013

SQL Server DB´s

For me as an old SQL guy it was also interesting to see how this all will be established in SQL Server Databases. In an enterprise environment a direct connect to SharePoint Databases isn´t used in general, two point why this is interesting anyway:

·         It’s interesting to see how the analytics database is used to plan you databases files, partitions etc.

·         Only reading from the database using given functions and stored procedures can of cause be an option.

Additionally configuration overview for your environment can be found here: MSSSearchAnalysisEngineConfiguration in your Search_Service_Application_DB

The interesting table in Search_Service_Application_DB is [dbo].[SearchReportsData].

In this table the data for the Usage & Search reports is stored. The data from yesterday is imported once per day from log folder under:  %InstallDrive%\Microsoft Office Servers\15.0\Data\Office Server\Analytics\EventStore

 

The table shows the result in my demo environment:

Using PowerPivot for Excel we can connect to that table and analyze the data:

Usage analytics

The usage analytics data is stored in the logging database. This article describes how to access the data using SQL Views: LINK

Using these views together with PowerPivot we can create our own reports. Here is an example for RequestUsager showing all sites request by “Administrator” for CA:

Also there is an official way consuming Usage Data directly from the logfiles and use it to feed PowerPivot hosted in SharePoint itself: LINK