okay i'd like to thank everyone for joining today we're going to be covering turbocharger experience with ai search we are going to need to speed through a lot of these topics pun intended uh so i hope that everyone's prepared to learn about ai search and how it can help you in your experience so to get started we do have to put out a quick safe harbor notice some of the things that we're going to be talking about i'm sure some of your questions will get into which direction we're heading in and what's going to happen next for ai search so we'll get to answering those questions and we may have some topics that we want to talk about some forward thinking forward-facing things but none of them are commitments necessarily i also want to mention that today's session is part of live on service now a curated event series with servicenow experts to help you get the most out of the products so we hope you join us again at another session posted a link uh in the chat so you can check up on future events on top of live on service now i do want to mention that we have a community and ai academy that focuses on some of these forward-facing technologies so ai search is one of many of these technologies and i would invite you to join uh the every other week ai academy sessions as well and some other housekeeping please stay on mute until you're you're called on there will be q a after the presentation but as questions come up please feel free to use the q a feature to ask them feel free to introduce yourself as well when you're asking a question we'd love to hear uh who you are and i appreciate all the where you're from during the opening here the session will be recorded and shared on the servicenow community forum after the session so you can always use this as a reference and then after the session ends we always ask you to please fill out a very short survey it helps us improve for future sessions and understand which topics are interesting and important to you a quick intro of me my name is gerard juan i focus on product success specifically for ai search and i'm part of the servicenow team before we get started we'd like to add a quick poll question basically asking how familiar you are with servicenow's ai search give everybody a little bit of time i see some answers coming in great participation today thank you all the votes are still coming in thank you and we are i'm going to end the poll right there it looks like uh many of you are some of you are here to learn but many of you have heard about it and done some research we do have two experts in the audience so i'm hoping that we can have a lively conversation about our expertise between the three of us so let's get started today we'll cover a few things as we get started with our ai search experience uh first and foremost and it's not covered here is we're going to just quickly talk about ai search give a quick demonstration afterwards we're going to talk about how we can measure ai search is it is it better than what existed before how do we know if we're providing a good experience what tools do we have to provide that measurement ai search uh number two uh ai search is it is it automatic what are my options to configure can i make it part of my experience uh it's something that we'll be going over as well and then if everything else lines up for you what do you need to consider before you go live what what do you need to know before you make the switch over to ai search ai searches is from an analyst and technological perspective we're talking about doing things that are actionable a lot of the predecessors and over time uh this is both the time and business value graph over time we've started with something that's very you know keyword oriented you type in a keyword you get a match uh that had evolved into a more semantic search based approach where you know we're doing yes you can do keyword searches but we have some level of understanding of what's happening with those terms with with those words that are in that search then we went into contextual search right as we got one across this this business value curve which basically helped us with some machine learning and then beyond that we had cognitive search right which which added some type of query intent capabilities now to make all of the to make actionable search happen we basically need to incorporate all of these items in order to to drive something um to make the end user perform some type of activity uh as part of that search result right how do we make this work i do also want to talk about a few uh metrics uh more forward-facing metrics or front-facing metrics than can you please mute if you're not muted great more forward-facing metrics so when we talk about forward-facing metrics we're really talking about how do we are we providing a better search experience and from a customer perspective for those uh customers that have implemented ai search they find that from the predecessor perspective from the predecessor's perspective there's really a double-digit improvement in relevancy most of the time we also have this average click position which is a great metric that we'll talk about a little bit later but basically on average where do end users click which search results do they click on and an average of two basically means that one of the top two results is the search result that an end user clicks on we also are seeing real deflection as part of enabling ai search so right off the bat we get nearly 70 percent of the users in that sort of self-service mechanism finding information that is available to them and then lastly i appreciate everyone everyone seems to be muted i i think we're okay now furthermore um there's a reduced setup effort um not only is it easy to configure but we also have the capability of improving the supporting technologies as part of of this as well so you can from a va set process setup perspective you don't need to incorporate as many nlu intents we can use ai search as a fallback which is a nice added feature looking at ai search from a holistic perspective we really focus on three areas when we talk about improvements or capabilities for ai search we are looking at the fact that it is natural we want to be able to interact with users and the mechanism that they're used to interacting it also needs to be smart right people are using the system and how can we learn to create a relevant experience for them going forward and lastly it needs to be easy to configure which is is part of the point on the previous slide in order to enable if we enable all of these things we basically have a system that is self-learning that we can set up in minutes that break silos and offers robust analytics and so while i've talked about ai search for a few slides i think it's also important to take a look at a quick demonstration so that we can go a little bit further here and make it real so in order to do that i will share my screen i will be showing you experience in an out of the box service portal uh but i do want to mention that we're available uh in any any portal that you may have that's based on the service portal including employee center and an employee center pro we also are available via virtual agent and mobile and i do want to bring up as well that we have the capability of starting an early access right now we have the capability of global search um improv including uh and including global search and uh the configurable workspace so as i click into this search box you see a couple of things the first thing you see is our autocomplete capability autocomplete for ai search is based on previous user behavior so you can see here recent searches and popular searches i'm going to go ahead and ask how do i set up automatic replies i can just use my my autocomplete for that the reason why i chose this query is because it also includes what we call a genius result which is that actionable item that we see in our search results from a very high level though we do have our uh navigation we start with our navigation tabs these are configurable you have the option of creating one of these for each search source that you include so you have the capability of narrowing down to specific sources of information for your end user whether that be a knowledge base article or catalog item or in our case we have some external content as part of this demonstration as well we also have the ability to sort based on most relevant which is using our relevancy out of the box relevancy functionality and would ultimately use that some of that ai tuned relevancy going forward we also have facet filters these are configurable out of the box they will include information about different filtering or faceting capabilities that we've configured for our source and it will be narrowed down to the total result set that we have so it won't be every item all the time it'll only have items that if you click on it you will ultimately get results the interesting thing here is that we have our junior salt this is called a q a genius result and this offers us a contextual snippet of content that exists within a larger article uh that exists within uh the servicenow knowledge base so here i asked how do i set up automatic replies and i got a staged option here it looks like this article does have what i'm looking for so i can always click to view on that article and get the individual information for setting up automatic replies in this case we also have and if i do a quick search here i'm going to do an oopsie search we also have you know on the on the idea of being a natural search experience we also want the capability to um automatically update based on spelling correction so in this case i did a search where i i fat fingered outlook but i'm still getting outlook based search results um using that standard format we're showing results for this if you really meant to search for for this other search you can but based on the information that's in the system we think you're probably looking for this um it's it's very good to have um derive what we call this this automatic spell check because it doesn't need to be a key or a term that exists in a dictionary somewhere it's really based on the content that you have so if you have unique service names as part of your organization or hr benefits then you can have then we'll get the spelling correction on that as well and last but not least i will do a last search for dev laptop and we can see here that dev laptop was expanded out to developer laptop or development laptop and so this is using a synonym capability and this is something that also is easily configurable we have the ability to make an actionable request based on this genius result i should mention that the number of juniors results presented on screen is also a configuration option we usually recommend between one and two genius results so with that brief overview of taking a look at some of the system level information that we have available as part of this demonstration i'm going to switch back to our presentation and let's talk about i am seeing some good q a where i think that will likely um get into some of that a little bit later great so the primary way to measure that we saw some searches being executed we saw some clicks happening so the primary way that we measure that is with something that is out of the box it's available in a store app and it's called ai search analytics it provides insights to drive higher self-service and productivity what we really want is administrators to be able to see what's happening we want to understand what the trends are over time and have an idea of you know whether we have enough coverage um as part of that experience so i'm gonna run through these slides um there are there are quite a few so we want to make sure we cover as many of them as we can first the ai search analytics is basically a one-stop dashboard to measure ai search there are really four categories of of information or options that we have as part of this uh the first is we want to understand the adoption right we talked about that a little bit we don't want to understand the quality of the search experience that we're providing we do based on some of these we may perform some tuning so there are some options to look at what's tunable and then there's some some things to look at if there are specific content gaps i do want to also highlight that this is all pretend makeup data um because you know we're not going to look at looking at live um searches or queries because of issues that may arise as a result of that so first and foremost adoption the first metric is search users so how many unique search users do we have performing searches on ai search this will help us understand if you know more or newer people are using ai search there's also a nice trend line or a trend value that will say hey you know since the last time or since the previous period here's the the rough change that we've seen over the course of that so it's a it's a nice it provides a nice understanding of what's happening i can tell you looking at metrics in july has been very interesting coming into august because all of the search users uh for many of the systems that i've checked are down but i hope that that just means everybody's taking a nice vacation let's keep that in mind as we go through the metrics total queries uh this allows us to understand how many queries we're supporting so you can roughly take you know a total number of queries divide it by total number of users and you get roughly how often people are coming back it's nice to have that again just as before you have some of the trending information there as well and finally average response time it's always good to understand how many uh how long it takes to send a query back um so it's it's always good to understand this there is a breakdown here where you can see the percentage of how often uh searches may be taking longer or shorter amount of time there are a lot of contributions to average response time it can include things like a high number of security permissions it could include things like if just very dense verbiage so it's sort of a nice quality metric but the actions need some additional investigation to really understand what's going on there searches over time right when i looking at real world customer data i can actually we can see when the weekend starts and ends uh which is great um oftentimes the highest search volume occurs um tuesday through thursday which makes sense so we have all of these different metrics that we can pull over time the nice thing is that it also provides sort of this this graphical representation representation of the number of queries that have been executed the number of research results that have been clicked so what is the success rate been as well as the number of genius results clicked so you saw that genius result pop up how often was that actionable based on the total corpus so it's pretty interesting metrics to have in track we also can take a look at genius results separately from everything else um so we can see the number of times that change results were triggered aka when they were presented on screen it's not going to happen for every search but when it does how often do users click on it so you can see sort of a weighted graph there to see the performance average click position i mentioned this earlier it is really the hero metric so what this does is it shows the average order of the result that was clicked uh so what that means is at 1.3 here this is demo data it's 1.3 is really good uh we will always target close to one we always want to get closer and closer to that one uh metrics which basic basically means that everybody finds the first result on on that um sorry everyone clicks on the first result thus finding information on that first result most customers and what we uh target for customers without any tuning is we target under three three or less is really what we look at so that's basically above the fold um as you're looking at search results uh so no need to scroll uh it also includes a breakdown so you can see how often you know they look they're looking at the top three or they may be looking at five through eight like did they have to scroll down to see it or are they looking at items past the first page just good information to know what's not pictured here is that self-solve rate um it's the percentage of searches that ultimately resulted in a click um what we are focused on is really the expectation is that it should be greater than 50 our or a very good metric is the metric we showed earlier at around 70 but the the target for untuned system is is over that 50 mark this helps with content gaps looking at top queries and tuning um so what we'll be able to see is what are people coming to the system and trying to find most frequently for all these subsequent questions for all these subsequent you can actually have a view all option so you can see not only the top queries the top five in this case and we can see much more going down that list same thing with queries with no clicks usually queries with no clicks mean that there is some type of content gap people have searched for something they've looked at the results there but they're not finding what they're looking for the nice part is that this is aggregated so you can see the top again the top five you can also see additional ones if you have a long tail of content of queries that have no clicks um there is an anecdote that i'd like to share where we did have a customer looking for a holiday calendar and their teaser texts just happened to have all the holidays through march and it was the beginning of the year um so it is something that sometimes requires a little bit of investigation if somebody's looking for holiday calendars and there isn't any clicks it may just be because you're providing the information they need right off the bat finally some content gaps or tuning potentially is the queries with no results nobody wants to provide gaps nobody wants to provide an experience that results in zero search results um so what we typically do here is uh recommend uh synonym expansion usually there's just a term that isn't being translated correctly by that i mean um somebody may be looking for an acronym or a system specific name or um information about uh a legacy system that has been replaced you may want to create a synonym there just to say hey you were looking for this legacy system but you were actually going to point you at the new one going forward so that those are different use cases for this and just as before you have the option to view all queries with no results and get a longer tail of those queries with no results again in aggregate form finally we also have this option of looking at the top clicked results uh so this is really a really nice metric it's really just kudos to either the knowledge team or catalog team for writing really good content right and it's a mechanism that we can use to understand the adoption of a new information that we want to bring into the system so again it provides a nice aggregation that lets you know what the data source was the total number of views as well as the average click position for that content so this is a good a good mechanism to improve the search experience or just look at what was performing well we talked about a few of the configuration options but what is what is possible to configure as part of the ai search experience so what should you consider configuring as part of the ai search experience what we'll talk about today is purely in this search profile part of the experience these these are all topics that may be familiar with folks that have seen the admin side of ai search but for those who haven't our configuration is is roughly set up like this and it flows from left to right um starting at you know your indexed sources so we're going to start by understanding which information should be part of the ai search experience from a holistic perspective then we move on to the search sources this helps us limit to the specific users that we want um that we want to to drive the experience for as an example this is a good one here um there there is a single knowledge base table um but there may be um specific uh pieces of information that we want to present in different ways uh from from hr or it then we configure this search profile which is going to be all the configuration options that i'm about to show you as well as basically offering some of these search sources basically a putting together of of the experience then we have our search application which has just some additional um application focused configuration so just to cover it really quickly you basically configure your facet filters as part of the search application as well as your navigation tabs those are the the primary items that you can configure as part of that experience and then finally there's the actual application that we're powering whether it be any portal now mobile as i've said or virtual agent now we're talking about quite a few different configuration options but i don't want us to be scared about this because these are all available as related lists in the search profile itself so you can navigate to the search profiles by going using your filter nav going to ai search search experience and then search profiles um the the image is a little bit small but you can basically see each one of these in a related list at the very bottom of the the page so search sources as we said what information is available in search uh synonyms these basically are word combinations that have the same meaning we also have the ability to add stop words which basically are common terms that have really no meaning in search uh we do ship with a stop word dictionary out of the box so you have words like ah and and right words that don't actually have any meaning as part of your search term that are abundant in the in the corpus but you can add additional ones if there are specific items that are part of your um content that are really don't offer any any any meaning in the search we also have the the typo handling that's our our spelling correction um these are generated automatically based on your search sources so it can be you know like we said it it's anything that is in your search content um can be direct derived and used as part of that that typo handling next is genius results we saw a couple of examples we saw a catalog in q a we also have people-based search that we can provide there as well and then we also have this option of result improvement rules so result improvement rules basically allow us to promote boost or block specific content this is often used to promote information during you know promote an open enrollment page during open enrollment and basically drive traffic to your open enrollment page and that can be based on any search query so you can say i always want open enrollment to come up on top boosting is similar where you're basically providing some context and driving information closer to that first result and then blocking content is just what it sounds like based on this query i don't want the search result to show up it's block is probably the least adopted right now because you can you can easily retire content that isn't relevant anymore but some folks still use blog content in the event that there's certain keywords that exist in that document that aren't relevant to to the end user experience so these are all of the options that are available as part of the search profile so quite a bit of power there another very common question when we think about configuration is what are the options for changing what this looks like so i'd like to talk through some of these options right now i'll start with one of our one of our queries here this this query is from a more hr specific use case you can see um there's a genius result the query was something like you know how do i book travel uh and so we have that the genius result page um as well as some of the organic searches uh as well as filters in your navigation tabs without going into all of the nitty-gritty detail there is quite a few there are quite a few items that you can change and update as part of some of these theming options for ai search uh including um font family looking at different colors for items looking at the the pill color for a selected facet understanding um what some of the pop-up options are for um the genius results uh highlight color changing border color changing border roundness for each of the search results or the whole card um so there are there are these mechanisms that you can do to change uh some of the theming um but you know from uh from our experience from the search perspective we're doing everything we can to continue sort of adding to what you can do from a theming perspective so what i'd like to do is actually run through a demo of how we can update some of these um this demonstration is based on a knowledge base article that we'll provide um in the in the chat here a little bit so i will walk through uh this option as well as the knowledge base so give me one moment i want to make sure that i'm still logged into my instance i am perfect so let's take a look i'm going to start and this is we're going to do this the um the quick way all of these options we're going to go directly into tables and add entries um all of these options are available through through the filter navigator or through other means but for the purposes of of demonstrating some of this i just want to run through um the quick way to show you how all this works so the first thing that we're going to do is add a new css and this css we're going to do two things we're going to do that those two common items the two common items are that we folks want to change the hit highlight color to make it closer to their brand standard so i chose the color hot pink just to demonstrate um that has changed so we'll we'll use hot pink for today i don't recommend using hot pink in your environment unless it's part of your brand standard so that's the first quick piece of css and again i'll we'll provide the knowledge base article that this is associated with um this will change the highlight color next um we are going to update a couple of classes this is to change um the search result border um to basically disappear right we want a lot of customers don't want the the box around their search results so this is wiping that out and then finally um if we're wiping out the search result box we also want to wipe out the the facet result box right the facet box uh because we want it to be a cohesive experience so we'll do the same thing on that side so with all of this in there i'm going to hit submit great and that is an indication to me that that has all worked and for those of us who are experts in css i don't know how many of us are but there's sort of this multi-step process where we want to make sure that the css is associated with the css include and i want to get my ai search common changes so now we've created a css we've created css include um and then we're basically going to add this to our brand new now our old theme uh the theme that we're using is the out of the box theme um la jolla and we just called our css include ai search common changes include so now all of that information is has been linked together and we can verify that by uh taking a look at our themes right ai search uses your portal theme and we have la jolla here and la jolla is now using our ai search common changes include so that's great everything there looks good so this is our previous experience right we have this hit highlighting here we have the the boxes on each of these search results as well as the box here so what i'm going to do is i'm going to do a cache clearing refresh browser cache and then we can see now that the outlines for our filters are gone our facet filters as well as the outlines on each of our results we do still have the hover over which again we can configure but i think the hover over helps in determining the selection so that hot pink color i think is really bouncing really well off of this so but it is showing that yes there is certainly a change here as part of this process great so with all of that in mind we saw at the quick demo we saw what metrics we contract we saw some of the configuration options as well as some of the visual configuration options that are available for ai search with all of that in mind it would be great to know we'd like uh to submit another poll uh it would be great to know uh if you are are planning or if this has changed your plans at all um in implementing ai search so we'll wait for [Music] some of these to come in it's great so it looks like a a fair amount of folks are are planning on going live with ai search that's fantastic uh we have we also have some that that are are thinking about it um and then we have actually quite a number as well that uh have already enabled the i search that's great i'm happy to have you all here fantastic well i think it's important to also understand um what you should consider when you go live and so to do that um i provide these quick these quick suggestions um first what we always recommend is enabling ai search in advance um so i uh this is one of those things where there's an automated process that occurs uh but we do want to make sure that when you when you are ready to go live when all of your configurations are there um that it is ready so we do recommend enabling ai search and production in advance of moving over all of the configuration options from your lower environments second identify internal content and the filters that you need to apply right we do have really great i think out of the box uh filters and configuration options um but sometimes they don't exactly align with what your business is looking for so it's always important to understand hey you know what are the items that are important to include as part of our search next we we definitely recommend installing the advanced ai search management tools this is an app that's an absolute must for anybody going into production and i think it's it's very helpful in pre-production as well so this includes capabilities not only like the ai search analytics that you saw in previous slides here but it also provides information like the search preview it's a great functionality that gives you information about what end users are seeing what synonyms are being expanded what stop words are out there so next we do recommend defining metrics that are important for use of ai search we do have some guidance on how you can provide some you can perform some benchmarking activities as well as up for forward-looking uh plans to include um a store app that assists in a lot of this benchmarking and providing guidance on that as well on top of all these once you're ready and you have your metrics we do recommend testing the experience by impersonating actual end users in in your system this helps with quite a few things it helps in understanding what information the end user is able to see um because we do adhere to all of the security practices in the servicenow platform so it's just good to have that baseline and that metric to go forward and then with all of these in mind we do recommend that that first week you watch that ai search analytics dashboard in order to identify and fine-tune right away in order to contin continue to increase the adoption there last but not least consider some of those result improvement rules maybe there's certain content that should be promoted certain access to people or certain people that should be seeing different types of content it's really helpful to get a metric uh or or get a barometer and set the pace um initially to get started with that uh i wanted to do a couple of quick call-outs before we go into q a i want to reiterate that today's session is part of live on servicenow that we recommend you know hey definitely sign up for upcoming webinars if you found this interesting please feel free to reach out and do that short survey i'm actually on the if you're interested in ai search and ai technologies um i know that i'm in uh ai and intelligence community fairly often and um the folks that have been helping out with q a are also being there often as well so we want you to feel supported as you embark on this journey we will provide and you've seen in chat some of the different options that are available different resources that are available to you i'll make sure that those are available after the session as well and with all of that in mind i'm happy to open up to q a i see a few questions that are out there already happy to discuss thank you very much okay great great um there's some great questions in q a um i would love to facilitate some of these so i'm going to start with a question that came in can you please talk about um the searching external sources beyond sharepoint can we search other sources you know google docs for example uh it's a great question uh from a servicenow perspective um with ai search we offer a few different mechanisms for ingesting external content honestly i think external content is a session in its own and where it is actually a session that we may be covering in a future ai academy so with all of that being said uh we do offer a framework to bring in external content um the framework is twofold there's the push method of a rest api where the source system would determine which information to send to ai search and we also have a pull mechanism via the integration hub and an ai search spoke so like i said i think there's quite there's a lot of detail to get into there but from a high-level perspective those are the the mechanisms that we use to integrate with external content and i'd love to get into the details in an upcoming session um i love there are quite a few questions i don't know if we'll get to all of them um let's see here uh our documents able to be pulled into ai search uh so there are this question is is has a it's quite a few sided so um there are lots of different mechanisms of including um i'm assuming like binary documents like a word doc or a pdf um as part of the servicenow platform the the primary supported mechanism is to attach documents to knowledge base articles so when a knowledge base article is has attachments the way that ai search works is that it actually indexes those documents separately and so what you'll get is as a search result you may get your knowledge base article and then as a separate result you'll get your attachment as a separate result if there are matches in each of those so we still apply relevancy the same way across both those options um [Music] uh okay yeah thank you thank you for the clarification talk uh uh clarification uh i from an out of the box perspective um sorry we got a clarification in chat um asking specifically about document management docs as part of a plug-in that's available in servicenow there isn't an out-of-the-box mechanism to incorporate those documents um as part of ai search i suspect that it may require some configuration i'm actually not certain so i think i would like to to take that offline okay uh and there was another follow-up clarification on attachments uh that i'd like to cover really quickly here um the uh the question is can you include attachments uh without necessarily having them separated out in search so that option is not available today the two options that are available are include attachments or not include attachments and so the include always has them as separate we are you know talking about forward looking we are making plans and designs around incorporating different options things like child nesting as part of that process as well so something that to look forward to but not something that's available today i think i answered a couple of the external content questions um there was a question that came in about um whether the clicks were everywhere ai search is used or only in portal i think this was in reference to our metrics so the the metric stream screen and actually one of the things that i didn't cover as part of that metric screen is that you do choose the portal or experience that you're you're narrowing down those metrics for so you have the option of choosing service portal or you know whatever application that um that you may have with ai search enabled so if you are you know if you have ai search enabled in multiple portals those will actually be presented in different dashboards within that ai search analytics so it'll be specific to each of those experiences ah very common question will ai search be available in pdis um the the answer is eventually it is planned to be available in pdis eventually it is a common question that we get um so we are really looking forward to providing that um we're we're still in the process of ensuring that we have the infrastructure available for all customers um in in our cloud environment and then from there we'll be expanding outwards so it's definitely something that we're looking into lots of questions about external content um i i think it is worse than a totally additional session so i am going to to skip over those and we'll make sure that everyone is aware when that session occurs we did get a question about whether this recording will be available for later viewing uh the answer is yes um i think it'll be available both at least in the community um and i there have been quite a few messages on from live on servicenow to offer that so i think the links will be available soon uh good question about the attachment that i missed um the the attachments uh yes the the interesting way is you can um configure the attachment card separate from from everything else so you can change some of the information that is presented um parent title is very specific to attachments and so um you'll always get it'll always be basically the parent you know document id um for you know the kbs it'll be kb whatever the kb number is dash the title of that parent um there isn't a way to customize that today um but it is you know we have gotten that request a couple times so i think it's something that we're definitely looking into especially as we um look at how some of the attachment functionality um going forward so uh with that i'm going to just update my i should have updated my um slide earlier um sorry i'm just catching up on additional questions here um there was a question about keywords being set for articles that show as featured content for users um i'm not sure if you're referencing something specific for ai search if you're looking at something in the legacy search but i do appreciate that you say that hot pink is awesome that was super helpful i think from from our perspective so the way that ai search works is we don't we don't focus only on uh a specific keyword field we actually look at the total content um that's available on the the article or or document or catalog so the the keywords don't drive um the relevancy experience as much as they would have in the previous experience so um [Music] what we're basically looking at is um the the total the the title right the the information that's in the the title or or name or short description depending on the table that's coming from as well as all of the content we do we are looking into for example in subsequent releases looking at the meta field again because that was something that customers really used a lot that ai search dropped for a little while but but beyond um sort of that that pure keyword matching we'll look at everything um so if there's if there's something that you're configuring an ai search that's that's behaving differently i'd be interested in knowing more about that definitely um there is a question about self-solve rate above 100 um we are looking um from a high-level metrics perspective we are looking at all the customer metrics as a whole to determine whether some of these anomalies like over 100 you mentioned um are signs of of good bad and different definitely inaccurate right um so i think the way that we're tracking um self-solve rate in certain versions of um of the metrics of the analytics just the i think the the numerator can be bigger than the denominator so that's something that we're looking into um i think i know the question asker so happy to follow up with you to make sure that you don't see that or to work with you if there are any updates [Music] i'm just following both meta there's a quick meta um comment in the chat uh yeah so you can you can always keep um meta for ai search um the the primary difference between the way that ai search first ai search doesn't index meta by default so there's a quick configuration you can do it's it's it's trivial um so you can you can always do that um and then the the primary thing to know is that when customers used to use meta pardon me um they would actually just do what's called meta stuffing uh thank you uh great great post from from loic um what what they used to do was called meta stuffing so um customers would just you know if something is about hr benefits they would just type in hr benefits 20 times in the meta field and that would show up every time you typed in meta benefit or hr benefits um the way that ai search works is we're really trying to figure out the aboutness and how relevant the information is based on the holistic content of the document so while you can include hr benefits or sorry you can include the meta field as part of that recall function it won't you know disproportionately boost that for hr benefits for for our example um it'll just say okay this is this is a match on hr benefits and if it is actually more more relevant from a relevancy perspective then it'll it'll bubble up to the top but it won't be because you stuffed your middle field and seeing we're at time i want to thank everybody so much for joining today i did see a lot of great questions in there that we didn't get to happy to chat more about it in future sessions uh or in talking with um and talking with you offline in the community so thank you so much and i hope that everyone has a great rest of the day