good morning good afternoon good evening everybody welcome to ai academy just before we get started just letting you know that we have additional resources available for you there is an ai and intelligence forum on the servicenow community where you can find a lot of content from experts and people on the field if you have any questions feel free to share them here there's always somebody to answer that that and then it it benefit the whole ecosystem so please do use that if you were invited to this academy by a friend or colleague just also you can go to our forum to sign up for any sessions in the future if you're watching that on youtube on the servicenow now community channel make sure you subscribe so you get the latest and uh all the updates on on all the content that is being published and feel free to share the recording with anybody you think would be interested today we'll be covering personalizing your ai search experience with tuning my name is loic sanchez and i'm an outbound product manager for platform ai products the agenda for today we're going to start with a quick overview of uh ai search and specifically tuning and uh that's only going to take about five minutes and then we'll jump in in a live building exercise where we'll see how to do that on a real instance and we'll get some q a throughout the session if you have any questions ask them in the q a on zoom and we'll be happy to answer those as we go along so with that being said let's get started and before we start configuring let's take a minute to explain what relevancy means and how search results are ranked so relevancy is like sorting for a table you can sort the records in a table by a date or text field well for search ranking or sorting is based on relevancy it means that a search engine calculates the relevancy score and then orders or ranks the results based on that score in old models the ranking is a static model where the score is calculated with a non-formula so for example for zing the old search engine of servicenow the relevancy is calculated based on the frequency of the search terms its sequence and the weight of the fields in in newer models this ranking is more dynamic and learns based on user behavior it means it's more adapted to long-term maintenance experience and search across tables for ai search the model that is deployed out of the box is benchmarked to be slightly better than zinc and in addition it learns over time based on the freshness of the records their popularity and also matches on title and body of the content as well as matches on the tags and popularity of those tags so all these user behavior that are used to to better the engine also translates into analytics you can access a ton of analytics by going on the user experience analytics dashboard and this is really the best way to look at what your users are looking for and make tuning changes based on that real production data it's the best way to look for areas of improvement and that really is the goal of tuning is to improve how fast a user gets your result by looking at the average click position and self result rate by looking at the queries you can find that maybe the content the user is looking for is missing in that case it's an indication that the content needs to be created but but when the content exists and the user just can't find it tuning can help it can help the users because the users create behavior maybe did not match the content or the content should be more relevant based on user context and that's what we're going to explore today but first a quick reminder of what was done in the previous session in the previous session getting started with ai search we showed how to set up ai search we showed how to make a table searchable by indexing its content income its content contents right with the indexed source and then how to filter that content in the search tools and we also mentioned that there's a search profile and a search application that makes search available on specific application well in the search profile there is a lot of settings that can help you personalize the experience and those are synonyms stop words typo handling and result in proven rules so with that being said let's go to an instance and start our life building exercise so today we're going to look at tuning some of the content in employee center so to to do that i'm going to start by navigating to my port source from my portals and open the airplay center record now for this specific port i see that i have a button here that says view analytics settings it means that the user experience analytics are enabled on that poll if that's not the case you should see a button that tells you to activate them first so that's a first step to do to enable analytics on that portal if i look at here it just see that i created that previously through the button it's all automated and now employee center behaviors and clicks are captured in my analytics and i'm going to use that later so what that means is that if i go to the user experience analytics dashboard i i should see a record for employee center so it's right there i can open that and so the first thing we see here is all the events that are tracked on that employee center but what i'm looking at specifically today is going to the search analytics dashboard so it's on the left panel here and i can open search analytics and what it does it opens the dashboard with all the analytics for ai search specifically for employee center so i see the number of users number of queries what's my average response time my trend of searches over time whether genius results are triggered or not in my average click position is something i want to make sure i keep under two or three meaning that users can get the content they need fast and my self-resolve rate as well and also i'm going to be looking at queries in one second so by looking at the queries with no click i can see what users searched and really couldn't really find any results so i could see for example that i have something around that or dev laptop so that's something i want to look at because i don't understand why they would not get a click here so to do that i'll go back to my search employee center for and i want to open my application first my application is actually going to tell me which profile i'm using on that portal so i know where to make my changes so i open my ai search profile for employee center and here i am so i have uh my search sources i have my synonyms my stop words typo handling genius result and result improvement rules and to do to start with my tuning i looked at my dashboard and now i'm going to use the search preview tool that comes with ai search the advanced management plugin i can open that in a new window so in that search preview tool i can test what my search results are based on specific profile so i select my employee center profile here and i'm going to see what happens when i type dev laptop all right when i type for dev laptop i can see that it's auto auto corrected into depth laptop i can see that if i navigate on the right panel here and click on the book icon here i see the spell check was triggered so dev laptop was auto corrected to that laptop and in addition to that i don't get great results and i noticed that there is a developer laptop catalog item here that really should be the results that i get first so what's happening here is that ai search doesn't detect dev as being the same as developer so in order to tune that i'm going to go to my synonyms here and create a new synonym dictionary and i'm going to call it my synonyms and i'm going to submit and i'm going to create a new term here and i'm going to say is when i type dev i want uh the engine to recognize dev as being developer and that that's that's where i create a synonym relationship between dev and developer and then i can publish that once i create a new synonym dictionary i have to publish my search profile again and now i have two dictionaries on my search profile and i included the term dev as a synonym for developer so now if i go back to my search preview here and type dev laptop now i get a genius result for dev laptop and the spell check was not triggered but i got a synonym between dev and developer that was triggered so that's great that means that my result now is more relevant so how does that look for my end user so i'm going to go as an employee of my employee center and just refresh and see what that experience would look like for them so if they were to look for a dev laptop now i would see that they would get a genius result so the most relevant results here is shown as an actionable call and they can directly start their request and that would make my and my analytics better uh so it makes my average quick position being at the top instead of at the bottom of the list so it means my user it's getting what they need to go faster which is good for the experience all right another thing i can do with my surname is use words that are specific to my organization so i can personalize my search experience based on what really is specific to my organization so for example our company use a software that is called global protect that is only used not necessarily by everybody but for for us it's a vpn software so what i want to do is that whenever i type global protect i want that to be a synonym to vpn why do i want that so if i go back to the user here and i want to tap i have an issue with my global protect i would see that i get a lot of results and almost none of them are related to vpn so it's not really helping the user there so to prevent that and make the experience better that's why i create a new synonym and i'm gonna say that when i search for global protect i want that to be a synonym for vpn and since i'm here i'm also going to say that global protect in only one word without a typo is also a synonym so that way if somebody types global protect without the space they will also be shown those results i made a change to my synonym dictionary so i have to publish it and that should be good though so i go back to my end user experience and refresh that page and my query is still global protect and what they get here they get results around the vpn which are way more relevant for that specific use case all right we talked about synonyms let's take a look at stop words i'm opening the default stop word dictionary so stop words are words that are ignored during my search because they don't have a relevancy so it's words such as your how so if etc by default that dictionary has 153 records or terms and it's usually pretty good if you need to add more you can follow the same steps we followed for synonyms but in that case i think for me it's enough they cover pretty much all my use cases typo handling so typo handling is that auto spell check that we saw previously and the way it's done it's based on indexing content from my index sources so if i have a close match between one of my sources it would suggest that a spell check uh by default you might have the sys user table here that is indexed and derived settings from if you're not using sys user just make sure you remove it and makes the spell check slightly more relevant but other than that here because i'm using knowledge in catalog items that's good for me not going to explain too much on genius results today but i see i've uh active true only for catalog items and that's why when i look for dev laptop i found i got a results there but let's talk about result improvement rules now so result improvement rules are ways to boost promote or block content based on specific queries or specific user context so let's look at an example if i go back to my user here and i'm going to look at the user profile i want to check which department they are from so my employee here is from the sales department and it just happened that my company released a new policy for traveling so they created a new knowledge article around travel and travel is limited right now but for salespeople we allow an exception the problem is that right now as any employee if i type for travel i'll get the request as a genius result which is good but then my knowledge article they concern they are related to domestic travel policy or travel expense policy which is kind of generic and then my travel exception for sales article is a lower ranked result here and i'm afraid my sales people they won't see that very easily so i'm going to create a result improvement rule to make that better so how does that work i'm going back to my search profile and go to result improvement rules and i create a new one and i'm going to give it a name exception for cells travel exception for cells and i'm not going to specify start and end date but you'll see they're going to be automate automatically populated and what i want to do is i want that rule to be triggered when somebody searches for something that contains travel and also i want that rule to only apply when the user department is sales so that way it allows me to i'm going to promote a content based on a specific user context i can see that uh when i create a trigger that is based on query i am there's a checkbox that appears here to see if i want to expand my query term to my synonym dictionary which uh is is good in that case so i want to submit that and you see the start and end data automatically populated and what i want to do is that when this specific set of trigger conditions gets triggered i want to promote a document and i'm going to type travel i'm going to look for my travel exception for sales document i'm going to click on the plus sign to add it to my list and then add document and close all right now i'm going to click on update and in that case my result improvement rule state is draft because i need to publish my profile and then it's going to be available and i'm going to go to my search preview to test that so we saw that as the end user when i typed travel previously the exception to the travel policy was uh four or fifths in the ranking if i type trial well first out so i want to check that rule based on user context so i'm going to use this advanced feature here on the preview testing tool here and i'm going to say if user department is sales so it's going to test the the search as somebody in the user in the department cells sorry click on search and now i see that my travel exception for sales is my first knowledge article that is returned which is better if i reload that page and do the same thing without any user context we still have the same for any other users which makes sense we don't want to promote that article if you are not from sales and so for the end user if i go back to that we saw that was previously what we were getting and if i refresh that page and type of travel and now i get my travel exception for sales so that way the content is promoted and it's more relevant for my users okay so now let's look at another use case i can also use result improvement raw to promote to block some content so what do i mean so as an employee if i type transfer because i'm interested in transferring to an internal position i see some results and one of them at the end is something around poor performance so as an employee i'm allowed or a manager i can see that there is some guide around poor performance that gets triggered when i type transfer it's not a great experience for my users they should not be looking at something regarding their poor performance where they look for transfer so i want to change that and the way i'm going to change that is creating another result improvement rule and with block and my poor performance article there and when the query is or contains the word transfer i want to block a specific document and that was my guide to addressing poor performance all right i want to block that when i type for transfer i'm going to update and i can publish my search profile i going i'm going back to my i i love testing my results first here because i get all uh different data on what's happening so i'm going to type transfer again and i'm going to see that i triggered a query rule here to block more performance so that's exactly what i was trying to do so that's good news i can debug or test that here and for the end user how does that look like so i'm just going to refresh and it's going to do the same query but with the new conditions so when i type transfer and i go all the way at the bottom of my list i don't see that article anymore so which is great i blocked that content so i made the experience a lot more in line with what my organization is trying to achieve all right that was it for the exercise