Microsoft Compliance Manager framework

Microsoft finally release the Compliance Manager: https://servicetrust.microsoft.com/ComplianceManager

This framework is designed to help companies getting and staying GDPR compliant or do other audits like ISO 27001:2013.

This article is focusing on GDPR topics.

Services included in this cloud service assessment

SharePoint Online, Exchange Online, Microsoft Booking, Microsoft Graph API, Microsoft Analytics, Microsoft Planner, Microsoft Stream, Office Delve, Office 365 Groups, Office 365 Video, Sway, Microsoft StaffHub, Microsoft PowerApps, Microsoft Teams, Skype for Business

Microsoft Managed Controls

Article 4 Number 8 of the GDPR defines that an entity that processes data on behalf of another is considered to be a contract data processor. Therefore Microsoft, with its services Office 365 & Azure is clearly a contract data processor within the meaning of Article 4 Number 8 of the GDPR.

Because of this there are also topics that Microsoft has to fulfill and the contracting entity had to check. These topics are aggregated in the “Microsoft Managed Controls” section of Compliance Manager.

The topics in this section are passed and tested by a third party independent auditor. We can get the details about every topic in the Compliance Manager as you can see in this example:

Customer Managed Controls

Not every GDPR article is about IT systems. Because of this not every article is covered by the Compliance Manager framework. In the section “Customer Managed Controls” Microsoft offers an audit tool that can be used to organize you GDPR compliance journey for Office 365 & Azure.

Features to organize GDPR compliance journey

1. Assessment a topic to a responsible person
2. Upload and manage documents
3. Track status
4. Test date
5. Track test result
6. Detailed description for each topic
7. Documentation about your implementation details
8. Documentation about your test plan & management response

Example:

You can use the Compliance Manager framework web UI to work with an auditor or you can also export the results as an Excel files.

Mapping

The GDPR is structured by the following topics:

• General provisions (Article 1 - 4)
• Principles (Article 5 - 11)
• Rights of the data subject (Article 12 - 23)
• Controller and processor (Article 24 -43)
• Transfers of personal data to third countries or international organisations (Article 44 - 50)
• Independent supervisory authorities (Article 51 - 59)
• Cooperation and consistency (Article 60 - 76)
• Remedies, liability and penalties (Article 77 - 84)
• Provisions relating to specific processing situations (Article 85 - 91)
• Delegated acts and implementing acts (Article 92 - 93)
• Final provisions (Article 94 - 99)

(only the highlighted topics are covered by the Compliance Manager framework)

Microsoft as a Software company is using different topics:

• Discover
• Manage
• Protect
• Report

Even the Compliance Manager framework is using a different structure. The framework is separated in:

•  Office 365 in-Scope Cloud Services (List of covered services)
• Microsoft Managed Controls (Topics Microsoft has to fulfill)
• Customer Managed Controls (Topics the customer has to fulfill)

So we need to do a mapping.

The following matrix is showing the chapters, the articles and the subitems covered by Compliance Manager framework. You can use this in you company-wider GDPR audit to get a clear overview of what is relevant in the context of Office 365 & Azure and what is covered by the Microsoft Compliance Manager framework.

File can be downloaded here -> LINK

Remarks:

Not every article need to be fulfilled by every company. In detail it depends on your company structure and what you do in detail with personal data. A general evaluation of which of these articles apply in a specific individual case, must be analyzed in a legally robust manner.

This article and the Excel Matrix was created to the best of the author’s knowledge and according to careful research. However it cannot and does not intend to replace an in-depth legal, process, and technical assessment.

Related articles: GDPR/DSGVO Field Guide for Office 365 & Azure

GDPR/DSGVO Field Guide for Office 365 & Azure

Starting May 25, 2018, the EU General Data Protection Regulation (abbreviated GDPR or DSGVO in German) will take effect, thereby becoming applicable law for all companies, regardless of size.

For the IT manager, this subject quickly becomes too theoretical and more than anything, contains too much legalese. The data protection officer and the compliance officer are usually a bit overwhelmed when it comes to the GDPR. The company management and the workers council put pressure on them and want to know if the relevant players are well-prepared or not.

This is the usual state of affairs when it comes to the GDPR within the company.

In this white paper, you will learn how to handle the subject with confidence and be well-prepared by the deadline of May 25, 2018.

Download the English version of the Whitetpaper for free: LINK

This document was created to the best of the author’s knowledge and according to careful research. However it cannot and does not intend to replace an in-depth legal, process, and technical assessment. Many thanks to Dr. Michael Rath and the team of Luther lawyers for the preparation and support.

Ab dem 25.05.2018 gilt die EU-Datenschutzgrundverordnung (DSGVO oder auch GDPR abgekürzt) und wird damit geltendes Recht für alle Unternehmen, egal wie groß sie sind.

Dem IT Verantwortlichen wird das Thema schnell zu theoretisch und vor allem viel zu juristisch. Der Datenschutzbeauftragte und der Compliance Officer sind meist etwas überfordert, wenn es um die DSGVO geht. Die Geschäftsleitung und der Betriebsrat machen Druck und wollen wissen, ob man bei dem Thema gut aufgestellt sei.

In aller Regel ist das die Situation, die sich zum Thema DSGVO in Unternehmen findet.

Erfahren Sie in diesem Whitepaper, wie Sie mit dem Thema hier und heute schon souverän umgehen und zum Stichtag 25. Mai 2018 bestens gerüstet sind.

Kostenloser Download der Deutschen Version des Whitetpapers: LINK

Das Dokument wurde nach bestem Wissen und nach sorgfältiger Recherche erstellt. Es kann und will jedoch keine fundierte rechtliche, prozessuale und technische Bewertung ersetzen.

Vielen Dank an Herrn Dr. Michael Rath und das Team von Luther Rechtsanwälte für die Zuarbeit und Unterstützung.

Interview zum Thema auf der Microsoft Partner Konferenz 2017 in Leipzig:

Ignite 2017 recording - explore PnP Partner Pack for IT pros admins and architects

Session recording at Ignite 2017. Me talking about PnP Partner Pack for IT pros admins and architects in the cloud and on-prem.

(Sadly bad video and audio)

My Ignite 2017 notes

Using site classification for SharePoint Sites

Site classification is a must-have when we talk about Governance, Compliance and also topics around GDPR.

Beside 3^rd party solutions focusing on site and content classification we have also some out of the box options and developer opportunities in Office 365 and SharePoint on-prem. Depending on if we are talking about classic SharePoint Site Collections or if we talk about modem Team Sites, being part of an Office 365 Group, we have different szenarios.

To create a new SharePoint site in Office 365 we know two different ways.

1. We can create a SharePoint Online Site using the SharePoint Online Administration. This will create a SharePoint Site based on WebTemplate STS
2. We can go to SharePoint Home and click “create” in the upper left corner or we can go to Outlook Online and create a new Group. Both will create a SharePoint Site based on WebTemplate GROUP

To provide a site classification solution for classic Team Sites created by option 1 we need to implement the following: Implement a SharePoint site classification solution. This works also for SharePoint 2013 on-prem. The article describes a full solution including policies for site closing and deletion depending on the classification setting. As you can see the article describes some steps to do:

• Define and set site policies
• Insert a custom action
• Custom site classification
• Add a classification indicator to site page

Using the opportunities we have with Groups and Group Policies some of these things can be automatically put to a SharePoint Site based on WebTemplate GROUP.

This video by Vesa Juvonen is showing the steps and the final results:

As you can see we need to create the site bases on option 2.

(Dialogs already including policies)

SharePoint Home - Create:

Outlook Online -> Create Group:

Final result:

Step by Step

To enable this functionality in Office 365 we need to set up an “Settings Object” and a “Settings Template” in Azure AD. To do this we can use the Azure Active Directory cmdlets for configuring group settings.

First of all we need to install the preview of Azure Active Directory V2 PowerShell Module:

Install-Module -Name AzureADPreview

To set up the site classification options and configure properties like ClassificationList and ClassificationDescriptions etc follow these steps also shown in Vesas video:

#Connect

Connect-AzureAD

Get-AzureADDirectorySettingTemplate

#Create

$Template = Get-AzureADDirectorySettingTemplate -Id 62375ab9-6b52-47ed-826b-58e47e0e304b

$Setting = $template.CreateDirectorySetting()

$setting["UsageGuidelinesUrl"] = "http://sharepointtalk.com"

$setting["ClassificationList"] = "Public, Internal, TopSecret"

$setting["DefaultClassification"] = "TopSecret"

$setting["ClassificationDescriptions"] = "Public:no restrictions,Internal:all internal users can access,TopSecret:only special users can access"

$setting["GuestUsageGuidelinesUrl"] = "http://sharepointtalk.net"

New-AzureADDirectorySetting -DirectorySetting $setting

#Check

Get-AzureADDirectorySetting -All $True

(Get-AzureADDirectorySetting -Id %%YOUR ID%%).values

As described in the video we can now use the CLASSIFICATION property to assign a site policy or any other custom action. Details about site policies are part of Implement a SharePoint site classification solution.

Here the script taken from the video to get the CLASSIFICATION property:

#Get PnP PowerShellOnline

Install-Module SharePointPnPPowerShellOnline

#Get Site classfication value

Connect-PnPOnline https://%YOUR TENANT%.sharepoint.com/sites/%YOUR SITE%

Get-PnPSite

$Site.Classification

Get-PnPProperty -ClientObject $Site -Property Classification

Overview of shared with Externals and shared Anonymous in Office 365

The GDPR highlights the need for protection of personal data held by organizations. To be able to do this Microsoft inverted a lot in new features and functions like the Office 365 Security & Compliance Center or the GDPR Assessment.

One of the backend systems helping to fulfill those regulations is the SharePoint Online Search Service. In the SharePoint Online Search schema, we can find two managed properties focusing on sharing and access from outside of your organization.

ViewableByExternalUsers and ViewableByAnonymousUsers

Both had the same setting: Query, Retrieve, Refine and Sort. So we can use them to create some reports based on search queries.

Personal overview

Office 365 let every user search in his SharePoint Online sites, OneDrive for Business files and also in Emails for content. In this scenario Email is of topic. But using this search function at the landing page of Office 365 a user can create a personal overview of content he shared to externals or anonymous.

To do this a user needs to fill in the following query in the search box at the Office 365 landing page:

ViewableByAnonymousUsers=true

In this example, I search for documents located in SharePoint Online sites or in my personal OneDrive for Business which are shared based on an anonymous guest link.

Using the query ViewableByExternalUsers=true shows me the files shared with external users through a sharing link that requires them to log in before they can view the file.

This gives a user an overview of documents he has shared from his OneDrive for Business with externals or anonymous. Because the URL is generic you can use this link for all your users and every user get his person overview: https://www.office.com/search?auth=2&home=1&q=ViewableByAnonymousUsers%3Dtrue

Also you can use this link to create a tile in the Office 365 App Launcher as described in the article: Add custom tiles to the app launcher

The result may look like this:

Team Site overview

Microsoft integrated a new out the box reporting capability in every Team Site. The article: View usage data for your SharePoint Online site is showing all details you need to know. There is also a new tab called “Shared externally”.

The article says: List of files you have access to that have been shared with users outside your organization through a sharing link that requires them to log in before they can view the file. Files shared with anonymous users or files available to users with guest permissions are not included.

To get a list of files shared anonymous in this Team Site we can again use the query: ViewableByAnonymousUsers=true followed by a path filter like for example: path:https:\\yourTeamSiteName.sharepoint.com.

Using Search Center to get an overview

As an administrator, you can also use the search center to get an overview of anonymous shared content or about data and also SharePoint Online Sites them self, shared to externals. The queries are basically the same and you can extend them with additional keyword queries properties.

For example, search all Office 366 Groups external users can access:

ViewableByExternalUsers=true contentclass:sts_site WebTemplate:GROUP

(Because of security trimming in SharePoint Search the user who runs the query needs access to all Team Sites to gets an complete report.)

Of cause there are also options archiving this using PowerShell for Office 365 Groups or using Reports in the Office 365 Security & Compliance Center. Using the SharePoint Online search gives you the power and flexibility to integrate all managed properties as metadata in you report like for example ViewsLifeTime, LastModifiedTime, CreatedBy or ModifiedBy. In addition you can easily scope your report to only show documents using the IsDocument=true query parameter or to focus to special Site Templates like WebTemplate:GROUP to only show Office 365 Groups Team Sites etc.

Using PowerShell to get the report

Using PowerShell to get results from SharePoint Online Search also offers the option to save the report as an *.csv file. To call SharePoint Online Search API using PowerShell and save the result to an *.csv file you can follow the steps explained by Prasham Sabadra in his article Office 365/Sharepoint Online - PowerShell Script To Call Search API And Get The Result.

This example is based on his description. The report is showing all external shared content and sites in an Office 365 Tenant and is saving the result to C:\Temp\ViewableByExternalUsers.csv

# add references to SharePoint client assemblies and authenticate to Office 365 site - required for CSOM   

Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"   

Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"   

Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Search.dll"

#Specify tenant admin and URL 

$User = "Admin@yourTenant.onmicrosoft.com"   

#Configure Site URL and User 

$SiteURL = "https://yourTenant.sharepoint.com"  

#Password 

$Password ="yourPassword"   

$securePassword = ConvertTo-SecureString -String $Password -AsPlainText –Force  

$Creds = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($User,$securePassword)

#client context object and setting the credentials  

$Context = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL) 

$Context.Credentials = $Creds

#Calling Search API - Create the instance of KeywordQuery and set the properties 

$keywordQuery = New-Object Microsoft.SharePoint.Client.Search.Query.KeywordQuery($Context)  

#Sample Query - To get the last year result 

$queryText="ViewableByExternalUsers=true" 

$keywordQuery.QueryText = $queryText 

$keywordQuery.TrimDuplicates=$false 

$keywordQuery.SelectProperties.Add("LastModifiedTime") 

$keywordQuery.SelectProperties.Add("ViewsLifeTime") 

$keywordQuery.SelectProperties.Add("ModifiedBy") 

$keywordQuery.SelectProperties.Add("ViewsLifeTimeUniqueUsers") 

$keywordQuery.SelectProperties.Add("Created") 

$keywordQuery.SelectProperties.Add("CreatedBy") 

$keywordQuery.SortList.Add("ViewsLifeTime","Asc")

#Search API - Create the instance of SearchExecutor and get the result 

$searchExecutor = New-Object Microsoft.SharePoint.Client.Search.Query.SearchExecutor($Context) 

$results = $searchExecutor.ExecuteQuery($keywordQuery) 

$Context.ExecuteQuery() 

#Result Count 

Write-Host $results.Value[0].ResultRows.Count

#CSV file location, to store the result 

$exportlocation = "C:\Temp\ViewableByExternalUsers.csv" 

foreach($result in $results.Value[0].ResultRows) 

{ 

$outputline='"'+$result["Title"]+'"'+","+'"'+$result["Path"]+'"'+","+$result["ViewsLifeTime"]+","+$result["ViewsLifeTimeUniqueUsers"]+","+$result["CreatedBy"]+","+$result["Created"]+","+$result["ModifiedBy"]+","+$result["LastModifiedTime"] 

Add-Content $exportlocation $outputline  

}

SharePoint 2016 – The Values of Hybrid, Cloud and on-prem

SharePoint 2016 – The Values of Hybrid, Cloud and on-prem

Hier das Video zu meinem Vortrag auf der Cloud and Datacenter Conference 2017 in München:

Delve and the Office Graph Inside Out Part II

In addition to my first post about the insides of Office Graph and Delve (Delve and the Office Graph Inside Out) this article is focusing on which signals are used by the Graph to generate the individual Delve experience.

Signals used by the Graph

You can find all the information you need about signals used by the Graph in this msdn article: https://msdn.microsoft.com/office/office365/howto/query-Office-graph-using-gql-with-search-rest-api

Based on this article we have the following Action Types:

• PersonalFeed
• Modified
• OrgColleague
• OrgDirect
• OrgManager
• OrgSkipLevelManager
• WorkingWith
• TrendingAround
• Viewed
• WorkingWithPublic

As you can see in the msdn article the list of signals can be dived in private signals and public signals so that data privacy is respected all the time:

https://msdn.microsoft.com/office/office365/howto/query-Office-graph-using-gql-with-search-rest-api#available-action-types

In addition Mark Kashman published an article on Microsoft techcommunity about Understanding security and privacy of Delve and intelligent experiences in Office 365. In this article, we can find the following diagram:

So we can extend the list taken from the msdn article to this aggregated version:

• PersonalFeed
• Modified
• OrgColleague
• OrgDirect
• OrgManager
• OrgSkipLevelManager
• WorkingWith
• TrendingAround
• Viewed
• WorkingWithPublic

• Member of
• Created by
• Shared with me
• Direct reports
• Public

Some of this signals are clear like for example Modified, Viewed, Created by, etc. some others are a little bit mystic like TrendingAround. We can imagine what TrendingAround means, but we cannot get an information about how this signal is processed in all details.

Anyway, it is easy to understand how this signals are used to generate the individual Delve experience.

The myth about the People suggestion in Delve

It is easy to imagine how content suggestions are generated based on signals. But one of the most asked questions about the Delve experience is about the difference between People list on the left and Related People in my personal Delve feet. Based on the signals list we can definitely get a better understanding about this. People on the left are other users we visited in Office 365 respectively we have clicked on their Delve profile. Related People are based on signals like “Member of”.

So for example if you are Member of

• Member of a Distribution List in Exchange Online
• Member of a Office 365 Group
• Member of the same Manger or “Direct Reports” entity

this is processed by the Graph to generate the Related People overview in your Delve feet.

As we can see also in this scenario data privacy and data security is respected by the Office Graph and Delve. If you are a “Member of” the same Distribution List or Office 365 Group, you can see all the other members anyway.

More details about this and also about compliance in Delve can be found in Mark Kashmans article “Understanding security and privacy of Delve and intelligent experiences in Office 365” I mentioned above.

Related articles:

• Delve and the Office Graph Inside Out
• All articles about Delve in my blog

Delve and the Office Graph Inside Out

Insides - Management - Compliance

While there exist many excellent blog posts, interviews and videos about Delve and the Office Graph from experts around the world, this post will be about Insides, Management, Compliance. We will also look at data security and data privacy aspects and concerns, not only from a technical perspective but also with regards to the new EU General Data Protection Regulation (EU Datenschutzgrundverordnung)

Insides

My experience with customers is that they need to understand more what the Office Graph is doing and how the results Delve is showing are build. I use in my sessions or in discussions with customers this picture from a Microsoft deck to explain a little bit the underlying fabric:

The Active Content Cache

•         Designed to enable near-real time updates at conversational speed (measured in seconds)
•          Contains most recently active items
•          Not designed to contain the full Tenant Graph, but rather the most likely to be relevant nodes and edges.
•          Every object has an expiration policy associated with it.

Tenant Graph Store

•          The full graph of all the nodes and edges within a tenant.
•          Optimized for analytics, not speed
•          Indexed to efficiently locate nodes and used to push nodes and edges into the Active Content Cache.
•          Because optimization decisions the latency of moving nodes and edges into the Active Content Cache cannot be guaranteed to be “conversational.”

Input Router

•         Directs the incoming edits to the Active Content Cache and Tenant Graph Store
•          Updates external applications regarding these edits
•          Powers the Conversational Experience

Workload Analytics

•         Specific to each workload, this is the piece responsible for reviewing local data and updating the Graph through the REST API. 
•         Only changes to the Active Content Cache or to Tenant Analytics are pushed by the API

Curios is that this is nothing you can change or configure. But to hear about the technology in the back gives customers a better feeling.

Management

I wrote a blogpost about how to switch to an opt-in like experience instead of the opt-out version. To be true this is only a workaround but customers like it. It gives them the changes to start with only some users in Delve to get more familiar with it. Opt-in as a default for Delve

If you don’t want a specific document to show up in Delve, you can create a HideFromDelve site column of the type Yes/No. This site column creates a new crawled property, ows_HideFromDelve, which is automatically mapped to the HideFromDelve managed property.

Compliance

We had an internal Yammer discussion with Mark Kashman about Delve Security & Privacy. Mark wrote the following statement and I asked him if this is good for sharing. Marks answer was: “Certainly OK to share the copy/paste'able section I wrote in the initial post of this thread.” So I will share this with you:

Delve is covered under the Office 365 Trust Center and meets all of the requirements of our highest level of compliance which Microsoft refers to as “Tier D” compliance, e.g., ISO 27001 and 27018 certification, SOC 1 and SOC 2 compliance. Delve is also licensed under the Microsoft standard Online Services Terms which include commitments such as the EU Model Clauses. This, too, applies to the Microsoft Graph - the underlying intelligent layer that uses advanced analytics to provide relevant, personalized insights via Delve and other user interface experiences throughout Office 365.

Office 365 customers own their Microsoft Graph data, which is stored in their partition of the SharePoint Online and Exchange Online environments. It, too, has the same data protection and security as other customer data stored in the same cloud services.

For users, Delve never changes any permissions on content or other information. Users only discover what they already have permission to see. Only users can see their private documents in Delve, unless they decide and take action to share them. Other people can't see each other's private activities, such as what documents they've read, what emails they've sent and received, or what Skype for Business conversations they've been in. Other people can see when others modify a document, but only if they have access to the same document. What you see when you open Delve is personalized to that user, and no one else sees exactly the same thing as they do.

It is possible to opt out of Delve and the Microsoft Graph at both the tenant level and the user level. Once opted out, users will not see the Delve tile in the Office 365 app launcher, and various services that surface aspects of the Microsoft Graph to provide intelligence throughout Office 365 will simply not appear, or revert back to previous non-Graph-based methods - i.e. search-based vs graph-based. One example, if you opt out, you would not see the new "Discover" tab within OneDrive for Business - yet the core of OneDrive for Business remains intact.

To learn more, please review these two important Delve security and privacy support articles; the first for admins and second for users: "Office Delve for Office 365 admins", "Are my documents safe in Office Delve?".

THX Mark!