Future of DocPush feature in SharePoint 2013

One of the most missing features in SharePoint Search is the option to push content direct to the index. In FAST ESP we have that functionality. Within FAST for SharePoint 2010 there was a tool called docpush.exe. Using this tool you can push content to the index. But this tool was not designed for productive use. There were several problems around this. For example if you push a document direct to the index using this tool and the crawler catches it also you have duplicates. During SPC14 the information came up that a push functionality for SharePoint 2013 search was already on the roadmap for SP1. Because of some unsolved problems it did not come with SP1. But this means it’s no longer a question if this function is coming – it’s only the question when it comes.

The function is no longer realized in a separate tool like the docpush.exe in FAST for SharePoint 2010. The function would be integrated as part of the SharePoint 2013 Search REST API. So there is a good changes that it also comes for SharePoint online. Locking forward what that all could mean for missing option to create own content sources for SharePoint Online. Stay patient. News are coming up soon.

Problems fixed by CU´s in SharePoint 2013 Search since release

March 2013 update
Server: KB 2767999
Foundation: KB 2768000

• When you perform a search on a SharePoint Server 2013-based server, you experience performance problems on the server.
• The output cache feature does not work on a page on a SharePoint Server 2013 site. This issue occurs when the page contains a Content By Search (CBS) web part.
• Assume that you use the Cross-Site Publishing feature to create a site on a SharePoint Server 2013-based server. When you perform a search on the site, the search results is greater than the expected number.
• Assume that you access a SharePoint site that is not in your trusted sites list. When you try to search an item on the site, you receive the following error message:
  • “The display template had an error. You can correct it by fixing the template or by changing the display template used in either the Web Part properties or Result Types.”
• When you try to perform a search crawl on a list that contains many columns in SharePoint Server 2013, you receive the following error message:
  • “The filename or extension is too long”
• Assume that you filter data in an Access form on a non-English version of a SharePoint Server 2013 server. In this situation, Full-Text Search in SQL Server does not work correctly. More specifically, the filtering operations are slow or return incomplete results.
• When you perform a search in a SharePoint Server 2013 search center in backward compatibility mode, the out of box (OOB) site refiner returns incorrect results.

April 2013 update
Server: KB 2726992
Foundation: KB 2751999

• Assume that you install multiple Search Service applications in SharePoint Server 2013. When you use a Search Service application to crawl a site, managed properties are not displayed in the Search Service application. Instead, the managed properties are unexpectedly displayed in another Search Service application. 

June 2013 update
Server: KB 2817414
Foundation: KB 2817346

• When you pass query parameters to query rules from Search Web Part on a SharePoint Server 2013 site, the search results are not updated. 

August 2013 update

Server: KB 2817616
Foundation: KB 2817517

• When you remove a SharePoint Server 2013 server from a SharePoint Server 2013 farm, the search admin component on another SharePoint Server 2013 server crashes continuously.

October 2013 update

Server: KB 2825647
Foundation: KB 2825674

• This update improves the stability of search components in large server farms.
• The language selection drop-down list on the search page does not extended fully.

December 2013 update
Server: KB 2850024
Foundation: KB 2849961

• When you add a friendly URL or a URL to a non-published item under Search Center Navigation on a SharePoint Server 2013 site, you receive the following error message:
  • Sorry, something went wrong
  • An unexpected error has occurred. 

Service Pack 1
Server: KB 2817429
Foundation: KB 2817439

• Search schema compression is now enabled by default to allow larger search schemas.
• Highlighting for FQL queries is now enabled for FQL as well as KQL.

Query SharePoint Search Analytics using PowerShell

Analytics have been completely revised in SharePoint 2013. Some of the core tech in the previous FAST release has been fused together with SharePoint enterprise search to create a new and powerful analytics engine.

The Analytics Engine is based on two main parts. We have Search Analytics and we have Usage Analytics features. Both parts have own reports which can be found under Site setting and as part of the Search Service Application in Central Admin.

Usage Analytics out-of-the-box report:

Search Analytics out-of-the-box report:

By using PowerShell we can query both Analytics parts and get custom reports.

Usage Analytics:

First of all we have to set the Search Service Application:

 $searchApp = Get-SPEnterpriseSearchServiceApplication

Next step is setting up a site scope for the report:

$Site = Get-SPSite http://intranet.contoso.com/sites/contoso/

Now we can execute the request:

$result = $searchApp.GetRollupAnalyticsItemData(1,[System.Guid]::Empty,$Site.ID,[System.Guid]::Empty)

If the report should not have a site focus we had to replace the “$Site.ID” part with “[System.Guid]::Empty”.

To focus the report data to a specific day or month we can use the GetHitCountForDay and the GetHitCountForMonth method.

First we had set up the data to which we want to focus:

$FilterDate = Get-Date "2014-01-20"

Then we can use the $FilterDate variable together with both methods:

$result.GetHitCountForDay($FilterDate)

$result.GetHitCountForMonth($FilterDate)

Search Reports:

Search Analytics cmdlets follows a different syntax. To get data from that engine we also had to setup the 

Search Service Application:

$searchApp = Get-SPEnterpriseSearchServiceApplication

Also in this case we can focus to a specific site using a variable:

$site = Get-SPSite http://intranet.contoso.com/sites/contoso/

The difference now is that we had to set up a date time value for the report. Leaving it blank is no option here:

$searchApp.GetSearchReport(1,[System.Guid]::Empty,[System.Guid]::Empty,[System.DateTime]::Now,$false,100)

That command gets the report data for the actual month: [System.DateTime]::Now . 

The next parameter which is set to “false” is the switch for getting data for the month = false or the day = true.

If we want the data for another month than the we have to set up a date variable and use it instead of [System.DateTime]::Now :

$FilterDate = Get-Date "2014-01-20"

$searchApp.GetSearchReport(1,[System.Guid]::Empty,$site.ID,$FilterDate,$false,100)

Getting farm wide data we also had to replace the $site.ID variable with “[System.Guid]::Empty”:

$searchApp.GetSearchReport(1,[System.Guid]::Empty,[System.Guid]::Empty,$FilterDate,$false,100)

The first parameter with the value “1” is setting the Event Type. Event Type 1 = views. The next parameter is the TenantID, followed by SiteID, date value, month / day switch and the max result value (100 means show me the top 100 search terms).

Create and use own usage event type in SharePoint 2013

The original technet article says the following about usage event types in SharePoint 2013

“Usage events enable you to track how users interact with items on your site. Items can be documents, sites, or catalog items. When a user interacts with an item on your site, SharePoint Server 2013 generates a usage event for this action. For example, if you want to monitor how often a catalog item is viewed from a mobile phone, you can track this activity. This article describes how to create custom usage event types, and how to add code to record custom usage events so that they can be processed by the analytics processing component. You can use the data that is generated by usage events to show recommendations or popular items on your site. This article also explains how to influence how recommendations are shown by changing the level of importance for a specific usage event type.”

So far, so good. Now let’s see how this can be done because of some code snippets in the original article won’t work very well.
In the following example we will create a custom event called “UseFulPage” and add a button to SharePoint to fire that event.

Step 1: create a custom usage event type

#Connect to SSA

$SSP = Get-SPEnterpriseSearchServiceApplicationProxy

#create Event:

#modify the $EventName varibale to your value

$EventGuid = [Guid]::NewGuid()

$EventName = "UseFulPage"

$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty)

$newEventType = $tenantConfig.RegisterEventType($EventGuid, $EventName, "")

$tenantConfig.Update($SSP)

#manipulate the weight of you event by setting RecommendationWeight & RelevanceWeight

$customEvent = $tenantConfig.EventTypeDefinitions | where-object { $_.EventName -eq $EventName }

$customEvent.RecommendationWeight = 10

$customEvent.RelevanceWeight = 10

$tenantConfig.Update($SSP)

#check if the event is created
$tenantConfig.EventTypeDefinitions | select eventTypeId,EventName | ft

The result should be something like that:

The new custom event had several properties and also 2 managed properties (UsageEvent1LifeTime & UsageEvent1Recent) which can directly be used in Search or Search Driven WebParts after the next analytics processing run.

Step 2:  add code to record the custom usage event
In the following script we need the GUID of the event. To get the GUID use this script:

$SSP = Get-SPEnterpriseSearchServiceApplicationProxy
$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty)
$tenantConfig.EventTypeDefinitions | select AppEventTypeId,eventTypeId,EventName | ft

To add the event to a SharePoint site I use a JavaScript snippet (thx Markus Alt).

1. Replace %EventGUID% with the GUID of your custom event.
2. Replace %URL% with the URL of the site where you place the event. For example: http://intranet.contoso.com
3. Use a ScriptEditor WebPart to host the script in your SharePoint site.

<script language="javascript">

function ToEventStore(url)
{
    alert("Useful Page Event recorded");
    ExecuteOrDelayUntilScriptLoaded(function()
    {
        var spClientContext = SP.ClientContext.get_current();
        var eventGuid = new SP.Guid("%EventGUID%");
        SP.Analytics.AnalyticsUsageEntry.logAnalyticsAppEvent(spClientContext, eventGuid, url);
        spClientContext.executeQueryAsync(null, Function.createDelegate(this, function(sender, e){ alert("Failed to log event for item: " + document.URL + " due to: " + e.get_message()) }));
    }, "SP.js");
}
</script>

<button onclick="ToEventStore('http://%URL%')">Useful Page</button>

Step 3: work with the event
Clicking the button brings up a message box which tells you that your click is recorded. That means that the event is send to analytics engine and is processed during the next run. So you have to wait until the next day to see the result. By the way: Analytics data is process only once a day. You can configure the schedule using the Set-AnalysisConfiguration cmdlet. Other important point is that only data from last day is processes, not from the actual day. Bella Engen descried in that post how to manually kick the Analytics so there is no waiting until next day to see the data: http://blogs.technet.com/b/tothesharepoint/archive/2014/01/21/modify-the-content-search-web-part-display-template-and-use-windows-powershell-to-start-usage-analytics-in-sharepoint-server-2013.aspx

Kicking the event of cause has an impact to the ranking of that site. Another option to bring the result out is to use the managed property “UsageEvent1Recent” in a Search WebPart as a sort option. Doin this you can build an overview based on search showing the most “Useful Pages” in you SharePoint:

Session preview video for SPC14

Uploaded the preview video about my session at this year SPC in Las Vegas: