For today’s intranets to be e�ective, they must bring together the information and tools workers need to get their jobs done. But our content lives in silos, spread across multiple line-of-business systems and cloud-based applications. It’s well proven that providing uni�ed access across these systems improves e�ciency dramatically, and also has a very positive e�ect on discovery of new insights and connections. The Connected Intranet is a pattern that works well.

BA Insight,Je� Fried

The Connected IntranetMake O365/Hybrid experiences complete using Cloud Hybrid Search and Connectors

Bringing External Content into Office 365 and Hybrid SharePoint

The Connected Intranet

Make O365/Hybrid experiences complete

using Cloud Hybrid Search and Connectors

Abstract For today’s intranets to be effective, they must bring together the information and tools workers need to get their jobs done. But our content lives in silos, spread across multiple line-of-business systems and cloud-based applications. It’s well proven that providing unified access across these systems improves efficiency dramatically, and also has a very positive effect on discovery of new insights and connections. The Connected Intranet is a pattern that works well. It is now possible to provide unified access across multiple repositories economically – leveraging new facilities in Office 365 and Hybrid SharePoint. You can deploy a Connected Intranet running in the cloud, on-prem, or hybrid. This white paper outlines how to include external content in O365 and Hybrid SharePoint experiences – without moving it or copying it. It also introduces BA Insight’s connectivity products, which provide high performance, secure crawling of content from many different enterprise systems into O365 and Hybrid SharePoint.

The Connected Intranet: Make O365/Hybrid Experiences Complete BA Insight

© 2016 BA Insight Page | 1

Contents Introduction ................................................................................................................................................ 2

The Challenge: Information Silos .............................................................................................................. 2

The Solution: The Connected Intranet ...................................................................................................... 3

Cloud Hybrid Search ................................................................................................................................. 4

Using 3rd-party Connectors with Cloud Hybrid Search .............................................................................. 6

BA Insight Connectivity for Hybrid SharePoint and O365 ......................................................................... 7

BA Insight AutoClassifier for Hybrid SharePoint and O365 ...................................................................... 8

User Experience ........................................................................................................................................ 9

Deployment ............................................................................................................................................. 10

Summary ................................................................................................................................................. 10

About BA Insight ...................................................................................................................................... 11

The Connected Intranet: Make O365/Hybrid Experiences Complete BA Insight

© 2016 BA Insight Page | 2

Introduction

On average, only 28% of the information that workers need resides in SharePoint or O365. The majority

is spread across multiple line-of-business systems both on-premises and in the cloud. Intranets and

Portals are expected to provide easy access to everything employees, partners or customers need. If

workers cannot easily access the highly valuable content that lives in multiple systems, they simply won’t

use the intranet – and they will remain frustrated by inefficiencies in their everyday work.

It is now possible to provide a Connected Intranet that bridges across multiple repositories economically

– leveraging new facilities in Office 365 and Hybrid SharePoint. This white paper outlines how to include

external content in O365 and Hybrid SharePoint experiences – without moving it or copying it. It also

introduces BA Insight’s connectivity products, which provide high performance, secure crawling of

content from many different enterprise systems into O365 and Hybrid SharePoint.

The Challenge: Information Silos

Our content lives in silos. According to IDC, information workers access an average of 6.2 distinct systems

daily for information to get their jobs done, with 40% reporting a need for 8 or more systems.

Corporations are experiencing explosive growth in the volume of information they have under

management, and they are also adopting new business systems at a rapid rate.

Moving content isn’t the solution – it’s a losing game. There’s a reason that multiple systems exist – they

are often specialized and valuable for important functions. Even if you could move everything to O365,

you would quickly discover additional sources, as cloud apps proliferate.

There’s a growing imperative to provide cross-repository views to employees, partners, and customers.

Unified views improve efficiency dramatically because users don’t need to navigate to many different

places to get the information they need. They also have a very positive effect on discovery – smart people

that see information from multiple sources together discover new insights and connections.

In addition, hybrid is the new reality for SharePoint, and Office 365 and Hybrid SharePoint are being

adopted rapidly. This brings many benefits, but it also brings challenges. Content can become even more

siloed as it is split between the cloud and on-premises farms. Multiple environments create complexity

and confuse users. Without unified access to the information they need to do their jobs, users lose

productivity and adoption plummets. Since the majority of information most enterprises use is resident

in systems outside of SharePoint and Office365, connectivity to external systems is essential to success of

enterprise portals and intranets – doubly so in Hybrid environments.

On average, only 28% of the information that workers need resides in SharePoint. The majority

is spread across multiple line-of-business systems both on-premises and in the cloud.

The Connected Intranet: Make O365/Hybrid Experiences Complete BA Insight

© 2016 BA Insight Page | 3

The Solution: The Connected Intranet

The Connected Intranet has become an imperative. Since users expect to have access to the information

they need, and moving it all one place is not a viable option, the only solution is to connect the intranet to

the key systems where the information lives. The information also needs to be presented together, so the

user can get at all the information relevant to the task at hand, in one swoop.

Enterprise search provides a wonderful way to unify information. By crawling content from multiple

repositories into a single consolidated search index, users can be presented with unified views quickly and

flexibly. Instead of making copies, the definitive information remains where it lives. The index provides

a quick way to find that information; when a user clicks on a result they are brought directly to the

document or record and can work with it in the source system.

Indexing connectors extract content from source systems and transmit it to a search engine for indexing.

Each enterprise repository typically has a specific way to extract content (access method or API), a

particular layout of content (schema), and specific security capabilities. Therefore, each type of system

may need a connector developed specifically for it. These connectors are difficult to build and maintain,

particularly because of security and performance considerations. More information about connectors

can be found in the “Secure Connectivity to Enterprise Systems” paper.

Figure 1 Unified views across multiple repositories using a consolidated search index

This approach has proven effective for years with on-premises search systems. A single searchable result

set across content from multiple repositories is a natural experience for end users, and the schemaless

nature of a search index eliminates many of the headaches of traditional information integration.

But there was just no way you could bring external content into the Office 365 index …. up to now.

The Connected Intranet: Make O365/Hybrid Experiences Complete BA Insight

© 2016 BA Insight Page | 4

Cloud Hybrid Search

In March 2016, Microsoft introduced a new hybrid search capability, called Cloud Hybrid Search, which

allows for an index in O365 to include content from SharePoint on-premises as well as Office 365.

Microsoft hosts a consolidated index within Office 365, to provide a unified view across on-premises and

online content.

Figure 2. Consolidated Index in the cloud using Cloud Hybrid Search (left) resulting in a unified result set

(right)

This capability is available using O365 along with SharePoint 2016 and/or SharePoint 2013. It is

described in detail in Microsoft documentation; a good starting point is “Learn about Cloud Hybrid

Search for SharePoint”

Cloud Hybrid Search has two main benefits:

1) It makes finding content easy, wherever the content lives. This is the essence of the Connected

Intranet.

2) It makes it simpler, easier, and less costly to run search. Infrastructure savings can be significant, since it dramatically reduces the number of servers required versus on-premises search deployments. The maintenance effort of running search is reduced because Microsoft is running this as a SaaS service. This makes the Connected Intranet more affordable.

Previously, the hybrid search experience (referred to as “Classic Hybrid Search”) involved two different

search indexes – an on-premises index in SharePoint Server, and a cloud-based index in Office 365. A

mechanism called Remote Result Sources allows results from one of these indexes to show up in search

centers in the another farm, but the results are not well integrated. Although there are still situations

where Classic Hybrid Search is required, for most scenarios Cloud Hybrid Search provides a better user

experience with a simpler deployment and lower cost.

The Connected Intranet: Make O365/Hybrid Experiences Complete BA Insight

© 2016 BA Insight Page | 5

Cloud Hybrid Search is based on a key element called the cloud Search service application (cloud SSA).

You set up a SharePoint farm running SharePoint Server 2013 or SharePoint Server 2016 and create the

cloud SSA on this farm. In most respects it appears exactly like the standard search service application

(SSA) in SharePoint – the administrative screens and tasks are the same. However, the cloud SSA does

not feed the index within its local farm. Instead, it parses the content and sends the indexable text and

metadata to the cloud, where it is indexed by your O365 tenant.

Figure 3 Cloud Hybrid Search uses the Cloud SSA to crawl content, parse it, and pass it to the cloud

Out of the box, the Cloud SSA includes the ability to crawl SharePoint content from its local farm, or

from other SharePoint farms within its domain. This lets you crawl and push content from SharePoint

Server 2007, SharePoint Server 2010, SharePoint Server 2013 and SharePoint Server 2016 farms into the

Office 365 search index. It also includes the ability to crawl file systems, so that content from file shares

can also be in the O365 index.

Cloud Hybrid Search shows up in many places beyond the search center:

Content from O365 and on-premises all appears in the compliance center for DLP and e-Discovery. This is because e-Discovery and Data Loss Prevention (DLP) are both based on search. Content indexed through the Cloud SSA is scanned with the same DLP rule as content in O365, and a single policy applies to both.

On-premises content is findable in Delve. The Office Graph is based off of the O365 index, and any recent content in the index will appear in Delve as a result. (Currently, the user has to query for it, and the content cards for on-premises content display filetype icons, rather than thumbnails of the documents.

Any Content-By-Search (CBS) Web part can include on-premises content that is included in the O365 Index. A CBS web part can look to the user however you like: as an ordinary list, as a grid of catalog items, or as part of a larger experience.

The Connected Intranet: Make O365/Hybrid Experiences Complete BA Insight

© 2016 BA Insight Page | 6

Using 3rd-party Connectors with Cloud Hybrid Search

Although content sources out-of-the-box are limited to SharePoint and File Systems, the Cloud SSA

provides a connector API. This means that 3rd-party connectors can feed content into the O365 index

using the Cloud SSA. For the first time, it is possible to put external content into the O365 index!

This mechanism can be used to incorporate content from many different sources into a single cloud-

based index. For example, BA Insight’s software uses it in several ways.

Figure 4 Feeding External Content into the O365 Index (BA Insight software shown in purple)

Content from a wide variety of systems can be crawled using BA Insight’s Connectors. Our Connector

Framework is the central hub for this content, and sends it to the Cloud SSA, which parses the files and

sends indexable text and metadata to Office 365. A service within O365 receives this content, processes

it for indexing, and sends it to the O365 Search Index.

Content from external systems indexed via the Cloud SSA is treated in the same way as SharePoint on-

premises content. This means it also shows up in the compliance center within O365, is visible through

Delve, and shows up in Content-By-Search Web Parts. This in turn powers portals, intranets, extranets –

and any and all search-driven experiences. It makes the Connected Intranet a reality.

Previously there was no way you could bring external content into the Office 365 index.

Now you can.

The Connected Intranet: Make O365/Hybrid Experiences Complete BA Insight

© 2016 BA Insight Page | 7

BA Insight Connectivity for Hybrid SharePoint and O365

BA Insight Connectivity products connects to on-premises and cloud-based systems so they can be

crawled alongside SharePoint content. In addition to content in the Microsoft suite (SharePoint,

OneDrive, Exchange, and Yammer), content from EMC Documentum, OpenText LiveLink, IBM FileNet,

Box, Jive, Salesforce.com, Microsoft Dynamics CRM, IBM Connections, and many more systems becomes

part of the O365 experience.

Our products make a seamless

user experience possible

through a rich index, including

consistent metadata and

content from enterprise and

line-of-business systems,

accessed with full security.

This is typically a single

centralized index hosted in

O365, dramatically reducing

the footprint and admin effort

involved with on-premises

search farms.

Where data residency or

content volumes prohibit a

single index, our Federator

searches multiple indexes

simultaneously and combines

the results into a single

interleaved result set.

Our solution gives customers a way to include all their important content in

Office 365 and hybrid experiences, independent of where it resides.

The Connected Intranet: Make O365/Hybrid Experiences Complete BA Insight

© 2016 BA Insight Page | 8

BA Insight AutoClassifier for Hybrid SharePoint and O365

Metadata is an essential element of the Connected Intranet. It drives navigation and findability, as well

as workflow and compliance. Yet the metadata on most content is poor or non-existent. When content

is brought together from multiple places, metadata is even more important, because it’s the way to put

information from multiple sources onto a common footing.

There’s well-established technique for getting common metadata across repositories. Access the content

while crawling it, use text analytics such as auto-classification to create or normalize metadata, and index

that metadata along with the content. With SharePoint on-premises, there is a built-in facility for custom

entity extraction and an API called the Content Enrichment Web Service (CEWS) which provides access to

content while crawling.

However, this technique is not available out-of-the-box with O365 or with Hybrid SharePoint. Microsoft

does not support CEWS API nor custom entity extraction in O365, and they are not available OOTB with

Cloud Hybrid Search. This is a significant issue as it blocks off a very important and proven technique.

To solve this, BA Insight’s Connector Framework has added a plug-compatible CEWS API, so that any

content indexed through our Framework can be processed by our Classification and Application software

– or any CEWS-compatible software. This means that common metadata can be added across all content,

to drive findability, compliance, and workflow.

For example, content from all sources can be processed using our AutoClassifier product. Term sets can

be synchronized across hybrid configurations using BA Insight’s software. This means that consistent

metadata can be applied to all content whether it is in Office 365, SharePoint on-premises or in external

systems.

Using this approach, all content is automatically tagged with metadata and processed for easy findability.

The machine-made metadata can also be used to drive workflow and compliance. For example, DLP rules

can use metadata created by the AutoClassifier.

Figure 5 Adding Content Enrichment to O365 and Hybrid SharePoint (BA Insight software shown in purple)

The Connected Intranet: Make O365/Hybrid Experiences Complete BA Insight

© 2016 BA Insight Page | 9

User Experience

Intranets and portals in O365, SharePoint Server, or both can now give easy access to everything

employees, partners, and customers need. For example, a user may need to access customer information

in a CRM system such as Salesforce.com, inventory information in an ERP system like SAP, and product

information in a custom database, along with documents in O365. Users do not need to know where

these systems are, nor do they need to sign into them independently. They simply see a unified view

across all the systems indexed.

Search experiences simply show additional content. The content can be presented together as a single

result set, or in separate tabs or result sources. No custom code is needed; the configuration is done with

standard SharePoint.

Figure 6 the OOB Search Center in O365 can include content from many sources

Any experience that uses the index or the Office Graph automatically gets access to external content. This

includes OOTB Search Results, tailored search experiences and search-driven applications, portal pages

using the Content-By-Search (CBS) web part, Delve, InfoPedia, and other next-gen portals.

Figure 7 Delve can be "lit up" with external content from many different sources

The Connected Intranet: Make O365/Hybrid Experiences Complete BA Insight

© 2016 BA Insight Page | 10

Deployment

With Cloud Hybrid Search, the search index is hosted in Office 365 and operated by Microsoft. This greatly

reduces the cost of running search and increase flexibility. Even in scenarios where some of the content

remains in on-premises indexes, footprint and operating cost is reduced.

Because the Cloud SSA requires SharePoint Server 2013 or 2016, it is necessary to have a ‘gateway farm’

hosting the Cloud SSA and the BA Insight Connector Framework. There are several options for deploying

this farm, including on-premises servers, Azure, or a SaaS service operated by BA Insight.

The BA Insight software is straightforward to deploy and can be on the same servers as the Cloud SSA, or

scale within the farm to very high capacity. Advanced configurations are supported as well, such as:

- Multiple SharePoint Server farms connected with one O365 Tenant

- Results from the O365 index in on-premises sites

- On-premises Indexes for selected content

- Combining Indexing and Federation

- Advanced Security

- Content scanning and inventory

Summary

The Connected Intranet unifies information and bridge information silos, thereby improving information

workers’ efficiency. A consolidated search index combined with connectors provides a wonderful, proven

way to achieve this. When that index is in Office 365, it is much simpler and less expensive to operate.

Microsoft’s new Cloud Hybrid Search provides a way to feed external content into the O365 index for the

first time. This paper has shown how Cloud Hybrid Search can be used with 3rd-party products to

incorporate content from many different systems.

BA Insight has worked closely with Microsoft to provide a portfolio of products that work in O365 and

Hybrid SharePoint, leverage the new Cloud SSA, and extend its capabilities dramatically. In this paper

we’ve introduced Connectivity and Classification products that work with Office 365 and Cloud Hybrid

Search. The resulting user experience provides a single unified view of content from multiple systems,

whether on-premises or in the cloud. This is a new way to light up experiences in O365 and Hybrid

SharePoint and provide natural and efficient solutions for users – resulting in a Connected Intranet that is

flexible and affordable.

“With BA Insight’s products, Office 365 and Hybrid SharePoint customers can now take

advantage of our cloud experiences for all of their content”. - Bill Baer, Microsoft

The Connected Intranet: Make O365/Hybrid Experiences Complete BA Insight

© 2016 BA Insight Page | 11

About BA Insight

BA Insight’s software addresses the Speed to Information challenge organizations face by quickly

connecting SharePoint users to the knowledge they need – linking key enterprise systems with an easy-

to-implement, use and upgrade software portfolio. Our software enhances SharePoint out-of-the-box,

increasing user adoption while decreasing user complaints. It replaces slow, high effort, and expensive

customized approaches with easy to use, off-the-shelf software.

Our software portfolio includes connectors and federation to provide secure connectivity to a wide variety

of systems; classification that increases findability using auto-tagging, metadata generation, and text

analytics; and applications with search squarely at the interface of users and information.

Hundreds of organizations and millions of users benefit from BA Insight’s software on a daily basis to

provide compelling intranets that people love to use including Australian Government Department of

Defence, CA Technologies, Chevron, Deloitte, Ford Motor Company, Keurig Green Mountain, Pfizer, and

Travers Smith. BA Insight is a Microsoft Gold Certified Partner, a member of the Microsoft Enterprise

Cloud Alliance, and an Elastic Partner.

Visit www.BAinsight.com for more information and follow us at @BAinsight.

Read more about our solution and our hybrid strategy at http://bainsight.com/hybrid-cloud-for-

sharepoint

BA Insight was awarded “Most Innovative Hybrid/Cloud

Solution” at the 2015 European SharePoint Conference for

our Hybrid Connectivity software.

Join the European SharePoint Community by following us:

For more FREE SharePoint & Office 365 content such as webinars, presentations, eBooks, videos & more check out our Resource Centre

To visit the Resource Centre please click HERE.

BLOG