hello everybody and welcome to AI Academy we're going to give another minute for everybody to join in and then we'll get started all right let's get started everybody uh so welcome to a Academy as always before we start we're going to share the Safe Harbor notice with you so everything we cover in those sessions is related to our existing capabilities that you can already use in the product but in case we make forward-looking statements or talk about the road map please follow the safe H notice and always check back with your accounting before making a purchasing decision in addition to the session today we have a few additional resources available for you we have a generative Ai and intelligence Community Forum that you can access with a simple link sn. works on this forum you have a lot of Articles and a lively community that ask and answer a lot of questions around our gener Ai and intelligence product so we highly recommend going to that site in addition to that on the right side we have a few links for specific content related to today session that we also encourage you to check out this session is recorded and then posted on our now Community YouTube channel where you can also find a lot of recordings for from other academies if you are new to AI Academy what's an academy it's content that we bring for you we bring you fresh idas to give you better understanding and practical guidance on our AI products as I mentioned this session is recorded and then posted publicly on our YouTube channel but if you are here with us today especially on that specific session today we encourage you to ask your questions and for that we ask you to use the Q&A panel on the zoom interface to ask your questions and we'll answer them today we're going to start with a quick overview and then we're going to have your questions answer today specific session ask the experts and we're going to spend about 20 minutes ask answering your questions and then save some time at the end for a wrap up and today specifically we're going to talk about AI search with now assist in AI search and we have a panel of experts that are here to answer your questions and with that I will hand it over to Sean to drive the session today if you're talking you're on mute thank you thanks Lu appreciate that good morning everyone good evening wh have year on the world I appreciate you taking the time out of your day to join us on this uh ask the expert session we have an exciting agenda today um but before we dig into answering questions uh we we we will go through a couple of slides slid that just set the scene uh just to remind ourselves as to what we have today with AI search and what we call now assist in AI search so for those in the audience that are new to service now and the now platform the service now platform helps customers automate and orchestrate key tasks across the Enterprise serving as a foundation for all workflows it's become increasingly clear that businesses must Embrace intelligent solutions to drive operational efficiency while at the same time improving employee and customer satisfaction AI search delivers a consumer grade search experience to employees and customers and empowers them with self-service answers to actionable information service now continues to drive platform Innovation and information for information retrieval and question answering while Zing search has been an integral platform search capability for many years years it lacks language intelligence needed for highly relevant search results AI search uses a combination of natural language understanding and machine learning to identify user intent to deliver always relevant results that users can act on right from within the search results window powered by the service now search Q&A llm Now assistant AI search uses retrieval augmented generation to generate concise actionable answers from the most relevant AI search results this is this is AI search with Next Level understanding that doesn't require retraining of the model AI search is now enabled as the default search engine in the service now platform it provides a personalized and consistent experience across desktop web and mobile and uses your existing service catalog knowledge-based articles and now platform data to quickly deliver the most accurate and relevant results anytime and anywhere search relevancy is automatically tuned by leveraging Machine learning to help you learn users behavior and predict the best answers so with that brief overview let's uh dig into some of the key differences between AI search and narrow system AI search will uh is this something you'd like to take on for us maybe just talk through uh what you you're seeing from customers and how we're we're positioning the two platforms yeah absolutely I can definitely talk about that so when it comes to the uh the previous AI search implementation um the AI search you know obviously it's able to retrieve documents from a variety of different tables and service now Etc uh it's able to offer us this modern search UI uh suggested content it offers the standard Q&A genius result uh it does have the multi-attachment view and then it also opens results in new tabs when we bring now assist into the picture however uh the main difference that happens is that we go from the now assist Q&A genius result or we go from this the Legacy Q&A genius result which is really just an extract of text from an article to a now assist Q&A genius result and so what this means is that the now llm is actually looking at the article and the user's query and attempting to answer the user's query based on the article so some great examples of where this might be able to help a user is uh if you have like a table that's embedded in an article uh now assist would be able to read through that table and possibly be able to give an answer to a user instead of just presenting them at table like the Legacy Q&A would be able to the Now assistant search would actually be able to give that conversational answer so um that's really the difference here is that that now assistant search is able to answer that user question um and also provide of course that actionable Q&A genius result card very exciting uh I mentioned earlier about machine learning relevancy uh so so Sheamus you're on the call can you uh maybe give us some some more insights into how that works absolutely happy to um so if we look here what we're actually looking at is our outof thebox relevancy model and the features so what's measured and what's important and we start by splitting these features actually into two categories for the a search relevancy the first categor is really based on a match so a match to the query itself meaning specific terms words or synonyms from the the user query compared against the the fields actually indicated here the other category is really the idea of a characteristic so things that are independent of matches on the specific article the characteris characteristics would be um article freshness or how recently it was created or updated and the other out of the box character characteristic is a popularity so how often a specific article is viewed or used based off of view and use count generally these two would be considered sort of tie Breakers unless there's a really significant difference in the value between articles um so actually sort of taking a step back and looking at the table overall these features are actually shown in order of importance of the to the relevancy model so there's a substantial weight applied to really a KB article number um sort of an exact match there assuming you don't have exact match turned off and then from there the other features are given weights sort of decreasing in magnitude so these features um are actually the what make up the relevancy model and are used to rank the organic search results and over time this model actually evolves based on user behavior that user Behavior being clicked on the search results themselves that will actually influence and allow the model to evolve and change the weights of these particular features and actually potentially add additional features to the model um based on the that user interaction or user Behavior so this is what is actually used to surface those knowledge articles to potentially answer a Q&A um or a question based on um that and provide an answer for the Q&A genius result very interesting and so how does this machine learning relevancy from AI search work in tandem with now assist in AI search another good question for sure and we have sort of a flow here that indicates basically that process so we obviously have a search execution and the retrieval of that top KB article and that top KB article is actually surfaced using that machine relevancy model that I just sort of spoke to the first thing that happens is it will actually check a multi-level cache for uh looking to see if a similar question has already been asked and potentially just respond with that that cached genius result if not what happens is based on that um that that request and the top KB article that actually gets submitted to the now llm to surface uh to surface an answer if possible if it does find an answer obviously it will store that in the cache to uh improve performance on subsequent requests of of similar um a similar question and then it'll obviously update the logs uh to indicate that that has been um surfaced appropriately and then merge that into our search results to be displayed in whatever experience you have configured whether that be one of your portal experiences or VA yeah that's great and so I'm wondering do you have any uh examples of how big a difference uh the performance is with and without the the cache turned on so yeah so there's there's certainly a significant difference there um our our request to the nowm will take um some period of time um but with with that cash turned on you're going to get responses uh in less than two seconds in your result set terrific uh will how about you how how are you seeing customer adoption and feedback and what sort of insights do you have from performance in the field yeah absolutely so working with a couple different large customers um they're seeing some really great performance uh I have a couple that are doing some very large scale testing and we've been seeing everything from the Now assistant search being able to quote links and context so bring them right into the uh the answer uh to being able to embed images uh or even being able to process and retrieve answers from tables so really really uh it's a much higher rate of occurrence so you're going to see this happen much more often than the Legacy Q&A genius result and you're going to see a much more intelligent answer when this does get one very cool some exciting stuff there I look forward to learning more about uh those hopefully in a case study or two so let's switch gears now uh one of the common questions we get and maybe this one would go to Trisha is how does virtual agent use AI search thanks Sean well so I work with just AI search and the virtual agent not even now assist so these are some of the quick tips that I share with my customers and so uh for the recording this is a quick snapshot of the slides we're going to go through first your AI search configuration is not as the same as the portal in which you are surfacing the virtual aging chatbot so that's key you can have a separate configuration from your portal so if you're expecting the same results you'll want to look at the configuration for that secondly um the virtual agent goes through a search sequence against the automation topics before it hits AI search so it's going to assess the user's utterance against nlu or keyword uh based searching for the automation topics and then when there's no match found then it calls AI search so that's something to consider um which will review the conversation structure for AI specifically there's a hierarchy to the search applications that are used in a configuration so if you could go back one slide to the summary and I'll come back to this you can incorporate AI search into a topic block AI search can be part of your fallback and that's specific to your app configuration but that's based on your conversational experience there's a hierarchy there I'm going to show you the G in a moment so conversation structure is key to a successful AI search so when you think about about how you're searching in your virtual agent you want to think about the topic elements AI search fallback at calling fallback now I covered this in a June 27th 2023 VA academy uh that you can that I really delve into it the thing I want to leave you with is that content is different from display and I've done a search on how many of our customers have activated the plug-in Advanced AI search management tools and it's not the same so if you're struggling with content I'm going to show you how to use search preview to see if you're surfacing the result you want to see if you get it there in your search preview but it's not displaying that's something that we're going to address in the search configuration so let's see what this all is let's go to the next slide so this is a brief snapshot of the conversation structure as I was saying we're going to go through your purpose topics that's um those Topics in the blue update equipment reset password manage meetings that's going to um your utterance when you launch the virtual agent chat poot is going to be matched against the keywords for that topic or your nlu intent if there's no match found then it will go into fallback and fallback is two pieces AI search fallback which I break into content and then fallback actions and so understanding the relationship of all these Topics in your conversation structure is going to be really helpful okay let's go forward the conversational experience drives those brings those topics together and so if you're newer to the virtual agent this is helpful to know your conversational experience can be unique across portals device types and we centralize all that configuration here in your custom greetings and setup and so here in my custom greetings and setup I can have a unique experience for my HR portal and that can be different from my phone experience where I'm integrating virtual agent with my ivr and then I have a catall which is the default chat experience there's more to it than I can cover in this section today but here's your takeaway this happens in sequence so HRC experience is going to hit all those conversation topics including AI search fallback is going to hit before say my phone experience and what you want to be aware of is if you could Advance there's an animation so if you hit the um next slide it should Advance there should be animation that highlights there you go so this guy is gonna hit first go let's go ahead and Advance this again can I do this um if you advance one more there's okay so the deliver this is an example of this HRC experience could actually be catching everything inadvertently I'm looking at the portal is ESC or and that's the key here or the device type is the web teams Channel slack Channel and what that means is it's not specific to just my ESC portal it's actually catching everything so something you want to be aware of if your search is not working the way you expect check your conversational experience and check your sequence it might be hitting another search mapping let's advance to the next slide yeah so just so one of the the common questions we get and you alluded to this earlier is you know how do I test my AI search configurations sure okay this is where I think this is a great tool to use just for testing content this is the search preview it's an additional plug-in available from the store and it's called Advanced AI search management tools you can tell if you have it if you go to your menu Navigator and you look for search preview if you do not see search preview you don't have this plug-in go ahead and grab this what this gives you is in this in the lower window you can see a way to test against your search profile now in AI search the Search application tied to your uh conversational experience search mapping or your portal search mapping is actually Associated to the content in the search profile so your search profile here in AI search preview allows you to see all your sources your scores how your results rank against each other and the panel on the right will show you your stop words your synonyms any sort of rules like your results Improvement rules or your Boost considerations that are in play this will be really helpful for you to understand how the platform surfaced the result it's surfaced what we're going for see the genus result right in the middle when I I ask who can get reimbursed for continuing education I want my genius result to surface and this is how I can indicate I can determine if how my result was found yeah this is such a powerful tool and one of the ways I've used this tool myself is experimenting with the caching that Sheamus explained earlier um you know running the same query and seeing how fast is the configuration for now assist and AI search without caching and then when you turn caching on and you ask the same question seeing how much faster that is so you know for all search admins this really should be their their go-to tool um you know to understand performance and and see the impact of any tuning they're doing absolutely great great example so let's go on to the next slide yeah so Trish can you explain the the search UI the evam no I'll give it my best shot the card how it displays so so I focus on the the content like am I finding my search results uh Sean Burton you had asked a question in the chat the virtual agent with AI search does not display KB articles only service cataloges I would look at my search profile I would use search preview first so do your testing there use your search profile that's being called from your conversational experience AI search so you want to make sure you're using the right AI search configuration for the right virtual agent configuration and then you're going to test it and if you're not seeing knowledge based in catalog then that's your search profile that needs to be um that needs to be reviewed for your sources and your sources need to be indexed your search profile needs to be published and your search preview is going to give you an indication of all that configuration now when we get into the display when we actually see the result on the portal or when we see the result in the virtual agent that's the evam cards that's your um that's the Search application configuration with your configuration bundle so there's two steps to display first sort options how are we sorting are we seeing the most recent search options first what are our genius results limits you can set that information in your search application configuration tied to this and I believe this is the next slide you're going to have your configuration bundle which is going to be stored as a bundle by experience so remember we started my my FAQs with you want to be aware of your conversational experience that's being called right now my HRC experience is being called from everything and my search mapping controls the content my ESC portal default Search application that has my profile that has my content my search UI evam configuration where the arrow is that contains the bundles for the cards that surface the results so that is if I'm seeing the results in my search preview but I'm not seeing it in my output you'll want to make sure you're in the right conversational experience and then you're going to want to troubleshoot the card here in this evam configuration that's all I got for the time we have yeah terrific that's that that's excellent okay so I think that leads us into our our final prepared question um will can you just talk us through what customers should do in regards to their data to ensure that they're ready to adopt now assist yeah absolutely this is a great question because you know as we're bringing generative II into the system a lot of people really think that it's just kind of a snap and it's going to be able to fix things which it very much is as long as it has good data to reference so um there are many points that we should consider when bringing gen into picture but uh one of the most important is quality of our document Corpus uh so checking that knowledge articles are kept up to date and that you have good processes around keeping those articles up to date uh that tasks have adequate documentation and that agents are interacting on the task or leaving notes on the tasks Etc um all of that is going to be really vital to ensuring that the information leveraged by gen is sound and that there is information to leverage at all uh data grooming and maintenance can really be sure to catch data issues before they become experience issues uh some examples of this in the actual product would include uh for now now assistant search uh if it were to leverage an article that was out of date U maybe you know a 5-year-old article that tells the users to do something um now you know the nowm is going to take that as gospel it's going to say this is the truth and so it's going to give that answer to user and you might get a poor experience um if it tells them to you do something incorrect uh with Now assistant it M we also see this when we're trying to do summarization or resolution note Generation Um if we don't have you know the interactions that an agent is you know completing and the things they're doing on a ticket Etc uh then when another T when that ticket is reassigned out to another agent and they try and get a summary they have to go and ask that person or they're trying to have teams chats all of that so really it's all about making sure that you have all the data that's required in the system and that it is you know up todate and and proper so that when generative AI comes into the picture it can really plug right into that and leverage it using our our very Advanced RG pipeline exciting stuff yeah we're seeing a terrific adoption and interest in now assist and uh now assist and AI search really does you know take search to the next level so you know I want to thank all my panelists for their time today um helping us just prepare you know answers and responses to the most common questions I can't see the the Q&A and chat uh but if one of you might just take a peek through there and are there any questions that have come up that we haven't addressed proactively just yet that we want to touch on in the last three minutes yeah I'm not sure if we have the answer to this one um but Mark andropolis asked is the cash local to the instance or is it part of shared infrastructure famous is that something that you have thoughts into so there's uh multi-level cache and I I do have to um probably follow up to definitively answer it but um the it would be the the level one cache is an in-memory cache so it would be relative to the instance and there's an L2 cache which is for the semantic support on the Q&A genius results and that's actually a table so that would be on the instance as well but I'll um certainly can follow up as well on that great is that something that uh should be asked and answered in the community that sounds like a good space for it yep is there anything else that stands out for you as good questions to ask and answer in this forum will yeah uh could you talk really quickly as to the road map for being able to use attachments within now ass system search as far as I'm aware we're on track to support attachments uh in the second half of this year that is awesome to hear and that's all that we have as far as things that haven't been proactively address terrific well thank you everyone for your time today uh that concludes this session uh we hope you will come back for for the next AI Academy thanks to all my my co-presenters thank you have everyone have a great day thank you bye right