logo

NJP

AI Search: External Content - SharePoint Connector

Import · Dec 02, 2022 · video

thank you and I we we are here uh together with a few other folks as well so uh really appreciate everybody that's going to help contributing to the Q a uh on on the servicenow side really appreciate that uh yes we are going to be covering the SharePoint online connector for AI search this is a continuation of I don't want to call it a series but it is sort of the next step in some of our external content conversations uh if you'd like to get a good overview of how external content works as a whole you can take a look at that previous video as as Nabil mentioned we are going to be chatting about basically how all these things work um it is going to be in an overview form so we're going to cover the SharePoint online connector for AI search there are many aspects of uh the the integration so we're going to cover an overview of everything today day and then as as there as the demands uh come in we will make sure that we have either separate videos or or other AI Academy sessions that focus on some of those additional points before we get started on the details uh just a quick uh point out here that if you ask questions that are a little bit forward-looking or asking for features that may come next um this is our our Safe Harbor burner is just acknowledging that they aren't hard commitments if we say we're planning on doing something going forward with all that being said uh let's chat briefly about uh when we looked at AI search we really focus on three specific pillars we focus on the fact that we want AI search to be natural that we want it to be smart and we want it to be easy and so when we think of the SharePoint online connector we're really focusing on this this idea that it needs to be sort of natural and integrated and we also are trying to make it as easy as possible so when somebody asks a question they should be able to get information through AI search regardless of the source and of course one of those common sources that we've heard from many customers is is SharePoint if we go a little bit deeper and we look at the overall architecture for AI search if we go left to right we're really focusing on there is content out there right um if you're using AI search out of the box you're already getting access to your knowledge base your catalog items now task tables and basically most tables that exist uh within surface now now what you're also seeing is this idea of external content and so we consider SharePoint uh a part of that that overall bucket but as you can see we bring information in regardless of whether it's from servicenow directly or as part of external content and then we if we go to the middle of this slide here and then we do all of the the processing that we would do for for any other piece of information that's part of AI search so we do that machine learning process we have nlu capabilities we do the natural processing on the languages that we support and then we also provide analytics on top of all of that so it's it's really part of that integrated experience and regardless of the source that it came from we can still power all of the experiences that are on this right hand side so Global search via the next Experience Store app now mobile portal any service portal based information virtual agent via AI sorry fallback and then of course the configurable workspace via again that AI search for next experience the reason why we take this approach of really pulling everything in and indexing it uh as part of of the the AI search index is that from our experience it provides the best performance so you're not making calls to SharePoint for example to retrieve search results based on your query you're actually indexing specific sites on SharePoint um so that you have it all available to you it also provides a unified list of results all relevancy ranked so you're going to receive one result set that has both your internal and external content as part of that we're going to do that machine learning process right when we when we talk about ml basically the AI part of AI search and then on top of that external sources also have the same configuration options via search profile as any of your other processes so if you want to for example highlight your shipment content or if there's a specific SharePoint um uh a document that you'd like to um uh promote then you can do all of these things as part of part of that search profile so basically provides that that unilateral um uh configuration event option if everything is together so that's that's our approach uh I do also when we go a little bit deeper we talk about the ship when they online connector uh we recently came out with a new version of the SharePoint online connector uh at the beginning of November and really the the key element here is that it offers uh security for the first time so in the past we've always said only index public sites everyone's going to have access to it that has access to the portal that you configure um there's no security but uh starting at the beginning of November we've added additional capabilities to include security so we're going to talk a little bit about that today we're going to touch on it um and I'll show you some of the configuration but we'll we'll follow up with a little bit more how all that configuration works on a later date when you have all of your information combined I just wanted a quickly sort of visualize what that looks like before we go into more of what the document flow configuration is so for an end user if somebody searches for example a pharmacy plan they have a unified experience as we mentioned previously so they basically have that that catalog item right to enroll in their their Pharmacy plan um they have it as that as a genius result and then you have your organic results underneath it that include that catalog item as well as you know if you have an external enrollment form in SharePoint or you have additional policy details that happen to be stored in SharePoint then you have that as well as part of this this unified result list but how does all that work so when you set up your SharePoint connector there are really two processes that are running um so the first one it's the one that takes the longest time right is that initial full crawl um what that will do is add all of your content your SharePoint content um to the AI search index if you've attended the previous external content session then you know that there's a series of steps that you need to do to to create that that search the index and the search Source but what we have is basically all that created for you so it follows the same process of having an external schema and creating an index source and having a search source so all that is is available it's all the same process as you might imagine and then once the full crawl is complete or in the process of it being completed you will have your um your SharePoint content available in the AI search powered experiences which we talked about earlier now the important part of any connector is being able to not only add information initially but to be able to go a step further and update that information as it gets updated in the source system so the other critical part of the the content flow is really the incremental crawl so what this will do is it will push updates to specific files to to AI search they'll also make sure that we're up to date on the deletes as well so we'll have access to both sets of information um and and make sure that we're updating the index so that it more closely um looks like what we have in trouble now you'll see that we do have um when we go through the exercise portion it is going to be a little bit of a cooking show style um walkthrough of what's going on there only because there are some steps that need to happen in advance right so we're we're going to have a partially baked cake but we'll we'll talk through what that process looks like anyway so from a very high level uh there are three personas that need to be involved in in this type of integration it really is an integration first and foremost you have your uh your Azure admin right there needs to be some user that is able to read the content that you'd like to index into servicenows AI search um then that user needs to be added to uh the appropriate group and have all of their permissions set up in order to to have that the right level of access then uh there's sort of this this hybrid uh service now SharePoint admin that basically is responsible for making sure that your servicenow instance can speak to your SharePoint tenant and so all of that process uh is is what has been set up already for us we'll take a look at what some of that configuration looks like um but that's that's basically what what needs to be done in order to set the foundation for uh ongoing indexing or ingestion and so once that connection is in place um what you'll see is that we can run a series of jobs that are scheduled so we'll do updates like we mentioned earlier and the full crawl will kick off as part of that process um and then finally um what we'll do is as the sort of servicenow search or portal admin we'll make sure that that SharePoint source and the the index sources and all of the other configuration that are part of the AI search configuration are ready available test it and make sure that we publish those uh updates to um the users right or the employees so that's a very high level look and we'll take a look at what some of those screens look like in a moment now I think the other important part and I said I would touch on it so I want to make sure that I do is uh really looking at some of the document security aspect of it so what when we say um document security we're really looking at the access controls that are on each one of these files or sites and so that information is coming in along with the standard process um that is part of that that full content crawl or the incremental crawl um the access control information is obviously a derivative of of the users and groups that have access to that information and uh when you when we walk through it you'll see that the years a new user mapping um table that's associated with the index source so that's where uh the individual mapping uh will start to make sense there so that's where we we do that from servicenow user to um you know specific user or group that's associated within SharePoint now before I go into the exercise um I'm not seeing any q a are there any uh questions that we can address real quick apparently there are no questions uh but if they're already a reminder to everyone please if you have any questions feel free to book them right now in the Q a section I think we just got one question Gerard right now it's in the chat it's um are restoring the SharePoint file content and servicenow that's a great question and I should have I should have answered that earlier uh so great great question Great lead in um so the way that this works is that during that indexing process um we're obviously going to have like the byte level information uh about the file uh so you know the dot docx uh file um what we end up doing uh is as part of the AI search process we actually crack open that file and only take the text out and then index that process and and the file is ultimately discarded right um so there isn't like a file storage or document management system that exists within servicenow it's only the contents of the file um that are are part of that process so just like all the other and and I think we might have started to address this in the external content webinar um it's only the contents of the file or the fields that are specified in the external schema table that is ultimately stays blank it just provides the shape for the type of content um that we get as part of the external content feature of AI search um we have another question does the service account we'll have an admin level privilege I yes yes uh so that that is one of the uh challenges I'll say of most Integrations is that we do require uh actually full read write um on on sites that we need to index and so that is because of just the way that SharePoint is designed we need it to do the incremental crawls and we needed to do some of the the checking of permission so it is a full read write admin user that we will need as far as justifying it I mean I think it is always a challenge right there each organization is a little bit different um but from from our experience it's really about you know understanding what information is important to have um and experience like employee Center or service portal and then that really being the justification around you know making sure that the right people have the right access to the right information so again it's a challenge and we understand that we're trying to make it easier but that's just a limitation of the integration great [Music] great great I think the last one is a comment which is great um I'm I'm thrilled that we've we've also added access control it is one of the things that many customers have been asking about for SharePoint so we're thrilled to bring it uh to you and and hopefully we'll have a follow-up session on on how um the nitty-gritty of how that works uh because we're I could see that we're short on time already okay great so I I would like to um with all those great questions uh in mind I would like to go through some of the configuration options as well as uh the connector page so that we we all know what we're looking at as we go through it so I'm gonna I'm going to advance here and I just need a thumbs up from uh someone to just make sure that that I'm on the store Page okay I I'll take that as a thumbs up okay great thank you okay uh yeah so uh the the SharePoint online connector is uh it's it's in a sort of limited access State um but it just means that you need to request access for it um we're happy to work with anybody it's it's a part of a broader team um on the employee Center side so happy to help anybody that wants to get started here the reason why I want to go to the screen and bring it up is because it also comes with um a full PDF that basically describes in detail in a very robust manner how to configure everything what it's like to connect your servicenow instance to the SharePoint tenant and it provides really an in-depth uh walkthrough of how to set everything up uh we're not going to go into this level of depth but I will be showing you today as part of the exercises is really looking at what the result of some of these configurations are so that you're not surprised uh when you go in uh to your instance if you request this so bearing that in mind uh let's okay I'll have to pause my sharing for a moment uh I apologize for that it seems like my session has expired and I will take care of that that's the downside of having things yeah it's always a challenge great okay almost there uh got a question about whether the document is available on the now documentation site or if it's something that that needs to be asked for I think with a limited access you may need to ask for permission for it I'm not as familiar with that process if you go to the the servicenow store and request access there um you will find that you can okay you will you will be able to find it and request access from there with limited access releases the documentation isn't part of um isn't part of the standard documentation yet um so it is in the process of getting there uh and then Victor asked another question about getting it for PDI uh not at this time actually AI search generally um isn't available on pdis so as we get closer to having some parity for pdis in general we'll we'll make sure that that gets added as well so thank you for that question that was a good one uh just going through uh knowing that that we have limited time um I want to highlight a couple of things I have the SharePoint online connector installed on this instance that's one of our our test instances and you can see a few things right um on the filter nav I did a search for SharePoint and I'm getting a series of information here right so we have the tenant information connections and credentials web Hooks and so on uh what I'd like to do is uh the tenants are not super interesting happy to click on them just for the sake of it um and just show you that there are basically two aspects of this and these These are highlighted in uh the documentation or that PDF that I showed a moment ago but basically the idea is that there there are two connection mechanisms one um to get sort of the the uh metadata information in one to get sort of the the raw um file information and you can see that from a connector perspective and I apologize for my pacing I'm just going to speed up a little bit um from a connector perspective we basically have this this connector that is that is responsible for a lot of the orchestration and the connector is associated with a site so the SharePoint online connector isn't going to connect to your tenant and just take all the sites uh as part of this you're basically going to set up a connection per SharePoint site that is of interest for any number of your portals and that's going to be the mechanism um that it's going to be associated with um and I before I go any further I'm just talking about the connector aspect there are a series of scheduled jobs that you can see here um that do everything from um in indexing the content to ingesting permissions to getting new users as well as incremental pulls so these are all things that are happening on an ongoing basis on a scheduled basis as part of that incremental or full pull uh the tenant config the tenant config isn't going to be super interesting um it's basically just a tenant ID and um so we got a question um from an anonymous attendee um but we basically uh the 10 the tenant config isn't super interesting it's really just talking about um what the endpoint is what the 10 ID is um and then what specific resources IDs are um these are really associated with the the connection that is created um between the um this servicenow instance and uh the SharePoint tenant so this is really as a result of doing some of the the authentication mechanisms and making sure that your your application or instances registered with SharePoint so with all of that configured uh what we get is is a few things um so we'll have as part of this process uh we we will have created an index source and you can see on my index sources that we do have this external SharePoint search and what we do have here is the table name and and if you refer to the external content uh webinar you'll you'll know that this table name is basically an empty table it is a mechanism to give us the shape of the information that we're looking for and the the big thing the big change when we talk about security is that this table uh this this sorry this uh index Source now has something called um external search user mapping table and so this uh without going into too much detail for today this user mapping table uh basically provides that that mapping between the users that are in the system for servicenow and the associated um the associated users or groups that are in SharePoint so it provides that mapping so that we can apply the correct security as part of that process so this is on the indexed source you can also see just as any other index Source you can see all the search sources that are associated with this so we have our SharePoint search source and then from here we would do what we would do with with any other AI search configuration where the search Source goes into a profile and it's associated with an application if we go through all this process and this is a little bit of a test instance so it may not be as interesting as others but if I do something like uh my my favorite query 3782 KB we'll see all of the documents that are associated with that and as you can see and one of the things that I wanted to highlight is just like um just like with other aspects of uh AI search any external content will have all the same features it will come with all the same features that AI search offers so that includes things like um the popular search or suggested searches as part of this process as well as things like spelling correction and so on I did have another instance ready that had some additional [Music] information on it just for some parity comparison this doesn't have the latest SharePoint connector on it so I didn't want to get too crazy but just like before right we have a separate tab as well as all the information that may have come from from that specific source so uh with that I know that we're we're very close to time um so I do want to thank everyone uh for joining today um and it looks like we do have a question one more or less question uh around connect rubber site or gonna be done at the side collection uh I believe it's a it's a per site configuration I'd have to check with the broader team um to see if site collections are supported as far as I understand it's on a personal basis uh and another great question sorry go ahead uh no it's a I think the question is uh can these slides be shared uh this session is being recorded and will be uploaded hopefully by end of tomorrow to YouTube and this is just a reminder to everyone if you missed other sessions you cannot go to our community and find the recording there uh for the rest of the sessions so yeah it will be uploaded really soon uh Gerard I want to thank you a lot and thanks everyone for joining today this was great session and uh looking forward for the more context on the AI search external sources that I'm sure we have more other use cases besides SharePoint so thanks everyone and have a great day thank you

View original source

https://www.youtube.com/watch?v=dPyXbOWh2Ug