Query Search (FAST) with PowerShell

A nice and easy way to send queries to you SharePoint Search Server (in this example FAST Search Server) using PowerShell. Of cause this is not an option getting results to work with if your scope is the content. But it is a very useful way getting statistics, summaries etc, Have a look at some property details or something else.

First we have to do is calling the needed .net class and create an object based on it:

#Calling KeywordQuery class

$searchSite = "http://intranet"

$site = New-Object Microsoft.SharePoint.SPSite $searchSite

$vc =

New-Object Microsoft.Office.Server.Search.Query.KeywordQuery $site

$vc.ResultsProvider = [Microsoft.Office.Server.Search.Query.SearchProvider]::FASTSearch

$vc.ResultTypes = [Microsoft.Office.Server.Search.Query.ResultType]::RelevantResults

In my case I enabled the FQL syntax and set some other parameters:

$vc.EnableFQL = $true # enable FQL

$vc.RowLimit = 2 # sets the limit of results

$vc.StartRow = 0 # 0 is the default

Next step is the query, in my case a simple query searching for the term “backup”

#Query / Result

$vc.QueryText = "backup"

$results = $vc.Execute()

write-host "Ergebiss:"  -F red

$results

Print out the results details:

#result details

$resultTable = $results.Item([Microsoft.Office.Server.Search.Query.ResultType]::RelevantResults)

$rows = $resultTable.Table.Rows

And getting the properties of each result:

#Each result with properties

write-host "Details:"  -F red

$rows

This is just interesting and nice, especially getting all the property from the “HitHighlightedSummary” and the “HitHighlighted” Property fields. But bases on this script, you can also do some other useful things, for example: finding out with filetyps that are in the resultset and how they are distributed

$searchSite = "http://intranet"

$site = New-Object Microsoft.SharePoint.SPSite $searchSite

$vc =

New-Object Microsoft.Office.Server.Search.Query.KeywordQuery $site

$vc.ResultsProvider = [Microsoft.Office.Server.Search.Query.SearchProvider]::FASTSearch

$vc.ResultTypes = [Microsoft.Office.Server.Search.Query.ResultType]::RelevantResults

$vc.RowLimit = 1000

# Query String: the "%%" means to search everything, the "-aspx -bdc3 –file" filters to near only documents

$vc.QueryText = "%% -aspx -bdc3 -file"

$results = $vc.Execute()

$resultTable = $results.Item([Microsoft.Office.Server.Search.Query.ResultType]::RelevantResults)

$rows = $resultTable.Table.Rows

$rows| Group-Object FileExtension| Sort-Object count -descending |Format-Table -AutoSize -Property Name,count,Size

$rows| Measure-Object Size  -Average -Sum -Maximum –Minimum

Output looks like:

Name    Count

----    -----

HTML       53

DOC        42

DOCX       20

XLSX       12

PPTX       10

PPSDC       4

VDW         3

XAP         2

XLS         2

ONETOC2     2

XML         2

VSD         2

TXT         1

            1

ONE         1

Count    : 157

Average  : 201299,133757962

Sum      : 31603964

Maximum  : 9711217

Minimum  : 0

Property : Size

Be careful with the “$vc.RowLimit” parameter, just getting the first 1000 or let’s say 10000 entities gives you a good overview and won’t stress you SearchServer too much…

Have fun playing around with it.

PS: Script based on a post from Harald S. Fianbakken, I corrected some syntax error and add some own ideas etc…

Using custom fields in search results

Find out the given internal name of the custom field....

SharePoint search crawler identifies automatically all fields in a given contend source. This will work for all contend types the SharePoint search server can handle per default or an Ifiler is installed. Of cause SharePoint user defined fields in lists or document libraries are crawled.

There is an MSDN article that describes how to use custom fields in core search result webpart: LINK. It’s just an easy doing, you have to put you custom field to the field list in the Display Properties node of the search results webparts and modify the XSLT transformation (XSLT) to display the custom property. Details see the link above.

But there a two little catches: first is that you have to map the crawled property that includes your customer field to a managed property to use it in search results. But that’s not a real big thing. You can do this using PowerShell or via SharePoint central admin. For example see my post: Bring up crawled contend to the user or this MSDN article: LINK. But to do this you have to find out which crawled property represents you custom field. For this you have to find out the static name of the field. Of cause there are several ways to reach this goal. I used SharePoint Manager; a free tool you can download from codeplex: LINK.

Open SharePoint Manager on you SharePoint Server, navigate to the list containing your custom field and click the field. In the right section you see the details of the selected field. In the cell static name you will find the value needed:

(this is SharePoint Manager on a SharePoint 2007 Server. It works similar in a SP 2010 environment)

In my demo case the field name is “2011 Fällig” and the static name is “_x0032_011_x002c__x0020_F_x00e4_”

Another way to find out the static name is to analyze the URL. To do this go to list settings -> click on the column name in which you are interested -> on the change column page the URL will ends with the static name:

… 176%7D&Field=%5Fx0032%5F011%5Fx002c%5F%5Fx0020%5FF%5Fx00e4%5F

This will work fine for simple field names, but you see that it can be horrible in case of complex field names.

So know we can use the static name to find out the crawled property. For example via PowerShell. If you use PowerShell it’s important to know, that the system puts an “ows_” to the static name of the field to name the crawled property. So our crawled property looks like “ows__x0032_011_x002c__x0020_F_x00e4_”. The PowerShell call in my demo case is:

Get-SPEnterpriseSearchMetadataCrawledProperty -searchapplication b835108f-5a59-4348-9cfb-c71adc1415b0 -category SharePoint -Name  ows__x0032_011_x002c__x0020_F_x00e4_

Another and of cause in this case easer way is using the search function for crawled properties in the search service administration section:

Now that we have the name of the crawled property containing the contend of the custom list field we can map it to a managed property and use it in SharePoint search for filtering, display it in search core results ore what else…

I´ve reached the next Microsoft certification level….: Microsoft Certified Trainer… jep :-)

A little bit more social FAST search

This article refers a blog post from Christan Glesser: LINK . In his article Christian describes that FAST search isn’t able to handle social tags as the std. search can do. The point is that social tags are not provides as refiners in FAST search.

There is an crawled property in FAST search called:

urn:schemas-microsoft-com:sharepoint:portal:profile:socialdata:socialtagtexturl

This property contains links to mySites from users who used social tags.

The property is unmapped per default. If you map this crawled property to a managed property (here you can find how to do this: LINK) you can do a search like: %managedProperty%:searchTerm

For example:

There is a social tag name “tag1”. If you search with SharePoint std. search for some contend witch is tagged with “tag1” the result looks like this:

If you do the search with FAST the result is the same, but there is no tag refiner shown (as Christian described in his post). What we can do now, if we have mapped the crawled property to a managed property and of cause do a recrawl, is using this syntax as search query:

socialTag:tag1
(“socialTag” is the name of the managed property I created for mapping the crawled property to it.)

The result brings the mySite from the user who used the tag “tag1”. On his mySite we find the tagged document:

Not really nice, and of cause horrible to handle for end-users. I think this is just only a theoretic solution, but interesting anyway. Maybe this can be used in a search driven portal or something else….

Bring up crawled contend to the user (map indexed contend to a managed property)

In one of my last post I wrote about extracting data via xpath out of the “hithighlightedsummary” field (LINK) - here I want to show another method to make contend that is in the search index usable for user experience.

I just want to use the same SQL Table as in my previous posts. Also I want to bring up the field “ApkBezeichnung1” (which is called “Task” in the BCS SharePoint List based on this SQL table in the previous post) to the front

 So the underlying and crawled data locks like this:

Now let’s use some PowerShell calls to see what the search server, in my case the FAST search server knows about this contend source. The crawled contend / crawled properties are normally sorted in so called Metadata Categories. This PowerShell call shows all existing Metadata Categories on your server:

Get-FASTSearchMetadataCategory

As result we get something like that:

We see that there is a category called “Business Data” witch contains all crawled Data coming from any BCS source

Next we want to do is showing all crawled properties in the Metadata Category “Business Data”

Therefore we use this call:

Get-FASTSearchMetadataCrawledproperty |ForEach-Object{ if ($_.CategoryName -eq "Business Data") {$_.Name} }

Another way to find out details of the crawled contend / crawled property we want to bring to the front is this call:

Get-FASTSearchMetadataCrawledProperty -filter apkbezeichnung1

“apkbezeichnung1” is name of the field in the SQL table.

The –filter parameter does not support wildcards. Matching is case-insensitive, and matches a string anywhere within the name attribute of the crawled property:

So we now know that the field is crawled and that the name of the crawled property is “read listelement.apkbezeichnung1”.

To use the property in SharePoint Search we have to map the property to an existing managed property or create a new managed property. I will create a new one with these calls:

1.       $v1 = New-FASTSearchMetadataManagedProperty –Name "BCSTask” –type 1

2.       Set-FASTSearchMetadataManagedProperty –Name “BCSTask” –Queryable $true –

StemmingEnabled $false –RefinementEnabled  $false

3.       $v2 = Get-FASTSearchMetadataCrawledProperty -filter apkbezeichnung1

4.       New-FASTSearchMetadataCrawledPropertyMapping –Managedproperty $v1 –crawledproperty $v2

The result is:

We now have a managed property. It can be used in SharePoint Search after the next crawl of the contend source. The managed property contains contend from the database column “ApkBezeichnung1”.

Examples:

Now we can do something like this and use the new managed property “BCSTask” direct in the search box: search term: “BCSTask:Dongel”