everybody I'm gonna get it started and then let our speaker do a great presentation today but as always before we get started just want to remind everybody that we have a ton of resources out there for you our Ai and intelligence Community forum is filled with articles and that's the best way for you to ask questions we have experts reviewing the questions and answering those so feel free to leverage that resource to access that we have a easy link for you it's simply SN dot Works slash AI That's our Ai and intelligence Community forum our recordings are available on the now Community YouTube along with recording recordings of all the other academies and additional content so that's also something we recommend that you can look at if you are interested in getting started with a broad understanding of our AI Solutions we recommend you take that course on now learning get started with AI Solutions that's on now learning and if you want to partner with servicenow in an agreement to share your data feel free to join our Global and Ai and Innovation program by reaching out to the email address on screen as always in those sessions we try to talk about features that are readily available in the product that you can already use but in case you ask about the roadmap or we talk about features that are coming later please keep in mind our Safe Harbor notice it just means that if we talk about forward-looking statements you always have to check with your account team before making a purchasing decision if you are new to AI Academy what is that so simply an academy is ideas that we bring for for you there are fresh ideas to provide you better understanding and practical guidance of our product here specifically our AI products and today we'll talk more about AI search specifically this session is recorded and then available in our YouTube channel on the now community that I shared just just two slides ago but if you are here with us today it's a great opportunity to ask your questions use the Q a panel on the zoom interface to ask your questions and we can answer those today during the the call in if we don't have time we'll get back to you and get you the answers with that being said we're going to talk about AI search uh specifically the AI search for virtual agent chatbot at knowledge so if you were to add knowledge and you used our mobile app with the chatbot you might have seen that it was using a pretty interesting search and so if you're like me and you're interested in knowing how that worked today it's super interesting session with Trisha that's going to walk us through uh how that was built and with that I'm gonna hand it over to you Trisha thank you very much like it's really nice to be here everyone my name is Trisha Cornish an organization and I focus on platform intelligence these are the tools that really facilitate digital transformation the virtual agent natural language understanding machine learning generative AI process optimization and AI search I definitely enjoy my job um our goals for the day I'm going to go through a quick overview of what we delivered for knowledge 23 give you a quick demo I have some examples throughout the PowerPoint presentation and hopefully I provide enough time to ask answer some questions really this is how we delivered a solution to help our attendees at knowledge and so I'm just going to cover this first knowledge was back in May held in Las Vegas and we had so many exciting messages to share the theme around knowledge and service now this year is yes whether you want to reduce costs or increase productivity yes we can do them both so it was all around collaboration inspiration celebration and Community um and so if you were able to go to knowledge you always know that there's lots of activity lots of information to share we try and help our attendees find their way around to different Labs different places in the venue because Las Vegas hotels are huge but the conference center in Las Vegas is remarkable and then there are so many places where we're sharing information from the hackathon to the meeting Center to training Labs as well as the exhibit hall how's the person supposed to find their way around well we delivered an event mobile app and it had all of it centralized it was really a great tool that attendees could download even before they arrived in Vegas where they could build their agenda of sessions they were interested in perhaps labs they wanted to check out we gave them maps to the venue there was also a game opportunity so you could kind of walk around and explore perhaps new areas that were new to you and score points and get points from the servicenow merchandise store servicenow doesn't do this alone we have our partners working with us so we have sponsors and partners that you can check in with as well as our sponsor ads notifications and this the virtual agent so if you think about it there was a lot of content in the events mobile app that I was not responsible for I did this with a team of people but there are some things that we wanted to provide through the virtual agent and from a product and capability perspective we wanted to see if our attendees would need to use a virtual agent or if we provided enough content in the mobile app so that it was completely self-sufficient it was kind of a little of experiment now real quick before we go into the solution um we have a poll so the virtual agent have you used a virtual agent maybe it's not in production maybe you're working through it um go ahead and just throw a vote up there if the virtual agent is new to you or if it's or if you've used it before specifically the servicenow virtual agent so let's just take a couple moments for that like let me um I'll let you tell me when we've got a good amount of votes yeah absolutely we already had seven almost 70 percent so it's it's going fast wait what's our what's the verdict fingers crossed I feel like yeah I think that sounds good about now 70 participated and 73 percent are I've used I've used the virtual agent it's pretty good that's fantastic okay great so um if you have used the virtual agent um and here's just an example of the instance we were using to prepare for knowledge there we go 74-26 that's a good amount that's a good amount um so for the for the uh participants that are new to the virtual agent I'm going to give you a little taste of it there's another Academy completely dedicated to the virtual agent but today's Academy is really focused on how we used the virtual agent to surface AI search results um so what we did is we built all of the virtual agent topics and within one of our instances that we made available and then we created a greeting we created some quick tips like what's the Wi-Fi which by the way was the most popular topic um how do people connect to the Wi-Fi and that's pretty much the whole engagement now now this this portal this was gone this it was inserted into an iframe that was part of the mobile app so for our testing for our builds we didn't need to do anything more than just make the chatbot available and then we did all the behind the scenes configuration in the instance but if you're new to the virtual Legion you might not be familiar with the structure of our conversations and the reason why I bring this up is this is this is going to impact the way we search for Content so in our chatbot conversation structure we have individual dual topic components that come together to surface a conversation like you just saw with my guest Wi-Fi we have a greeting which I can completely configure we have topic discovery which is show me available topics and then we have what we call purpose topics in my example that I just showed that would have been the guest Wi-Fi these are the virtual agent guided back and forth for say update my equipment precept password managed meetings now if this is all I have in my virtual agent I might also have my catalog and my knowledge base available and that's I'm going to make available through what we call our fallback and in for me I've crafted my virtual agent Solutions so that I search that content when a user asks a question like request a laptop so I don't need a purpose topic around catalog request I can tell my users hey team members if you need to request something from the catalog go ahead and just type in request a new parking pass and we'll find it we won't find it in the purpose topics right these are my automation topics but we will find it in the content search now in our fallback we have content search occurring first and then if a user asks something just crazy or maybe we don't have anything in the catalog that's being used in AI search and there's no purpose topics then we have what I call the fallback action what should the virtual agent do if no content is found and if you think about it in an um in an I.T environment or you know an organizational environment where users are asking for it support or HR Support this makes sense I want to inquire on my benefits I want to reset my password I want to order a new application license all of that can happen uh what's for dinner on Saturday well is this something we need to know but if no content's found you use or do you want to create a ticket do you want to call an agent search topics or end so from a um an organizational perspective this works really well and I'm going to go to my next slide so as an example um here's just uh a self-service portal that I use to demonstrate to my customers where I'm provided a custom greeting uh I can tell them how to use the virtual agent I can ask any question so when will I be able to retire as an example that could be specific to my benefits inquiries that could be specific to just you know any sort of information so I'm going to display information from my knowledge base or catalog this is then if I say I need more help I can do some follow-up so I haven't found any content or um actions and so now I can create a ticket or send it to a human so this is what the experience is traditionally that I work with a lot of my customers with but here's the thing at knowledge we're not in a traditional environment right I've got attendees from all over who have a variety of experience with service now we have some attendees that have never seen servicenows capabilities in the platform so I have to be aware of the users that are new to the servicenow ecosystem to knowledge there's no historical information available whoops there's nothing for me to build actual virtual agent topics on additionally it's a three to five day event there's really like once it's out there I'm kind of living with whatever I delivered I'm also working with unauthenticated users so again going back to a variety of skill sets I also um have to just keep in mind that it's unauthenticated information that I'm making available to our users there's also no follow-up and assistance so whereas in my organizational chat bot I could say do you want to create a ticket do you you want to speak to an agent I can't do that here so I'm going to be changing my virtual agent so that I can make it more self-service friendly in this conference environment and finally and this is no small thing the mobile app which I was only a small part of it's pretty amazing it's already got venue information and and session information so I had to Think Through how I wanted the virtual agent to exist within this mobile app and so I thought of three applied principles so the chat experience is really just q a we have to make it very quick for users to ask a question and get an answer from Hello to goodbye we have to make it fun and exciting and really focused on the knowledge 23 experience so in the organizational chatbot environment where I might be delivering a self-service experience or across my Enterprise it's a little bit different here at knowledge it is just q a I want so in that case I need to make search work first and if I make search work first that means I have to surface the right content or I have to direct users to a helpful resource right it's it's not counter to what's going on at knowledge I have to make it additive so um because the mobile app even though my virtual agent is embedded in the mobile app um I have to make sure that I'm not creating a conflict of information so whatever information exists in the mobile app I'm going to surface some of that in my virtual agent and if the user can't find any information around there I need to direct the user to an ambassador to a real person so I have to change my fallback action to direct the user to go ask an ambassador for help or go to the help desk and not this is this was actually new to me in this experience not all users are equal in their work habits so we might have very experienced servicenow participants who like to navigate versus just asking open-ended queries so I needed to make sure that I provided an experience for all types of users and all experience levels and so that's where I came up with this knowledge23 chatbot experience so quick experience one and done questions asked and answered we're going to make it completely conversational and we're going to make sure the branding reinforces knowledge 23 we're only going to display publicly available information so anything I put in the knowledge base any Uh custom information that I'm going I'm going to surface from tables that I made as part of my AI search resources are publicly available information where I have set my security so unauthenticated public users can review it I also need to make sure that the information Works in a primarily mobile environment so we did have iPads at knowledge and we had kiosks but really people might be picking this up from their phone and so the content that I surfaced was pretty summarized had to be short and sweet with no links that clicked anywhere else so I changed the way my conversation structure looked so as an example I have a custom knowledge 23 greeting I have still of my topic discovery which I'll show you in a minute but then really what I got to was my knowledge 23 content search and this is really where I spent a good amount of time preparing my knowledge 23 sources for venue information sessions and speakers I made sure that I could find faq's product areas because the team that I was is working with were providing me some sample questions based on experience most people ask when they approached the help desk in Prior years what's the Wi-Fi when are the exhibit hours how do I find itsm what is itsm there's a host of different questions we started with that I made sure I could find here now here was the interesting part of this my fallback though like I said because I can't do any follow-up I had to make sure that I changed this to find an ambassador and I think this is key understanding that fallback is actually a two-step process and then of course my closing we hope you enjoy knowledge and so here's a couple of examples that I just showed you in my Wi-Fi example we have our custom greeting which I tried to guide users who might be familiar with a chat bot how to query some helpful information here's how you can ask for sessions for speakers for venue information and here's how to ask FAQs can I buy a day pass when is One Republic playing things like that promoted topics are a feature in the virtual agent that helps us surface the most helpful topics now we just know from attending conferences and Live Events the most common question is how do I connect to the Wi-Fi or you know what's happening today that I really need to pay attention to so I did create a couple of topics so I could surface Maps so I could present today's Keynotes or so that I could answer the question for the Wi-Fi where I did need to point a user to an ambassador I tried to make it as visually appealing as possible and say look for this shirt and then of course identifying in the clothes what our hashtag was so we could share on social media so that really was the goal within our virtual agent now here's the second part that I really want to jump to and this is make search work first and when we make search work first what we need to do is make sure the content of search is more than just knowledge base so I had vocabulary custom tables for my venues and speakers I had um I had knowledge-based articles for my product pages and I pulled that together in my AI search resource and I'm going to pull this up for you I was able to accommodate additional search criteria using meta tags synonyms and stop words and what I mean by that is for example we Pro in the venue we provided a meditation room a quiet area a meeting Center in addition to just identifying locations for training labs and so being able to differentiate between the meeting Center and a place to meet for um for the hackathon query might be the same where do I go to meet but actually we want to direct the user to a very different place so I'm going to show you those tables in a moment but as an example we also had to consider different ways that users might ask for when is Dale Stubblefield speaking who is presenting at the hackathon speaker versus presenter training versus lab I had to make sure I accommodated those synonyms because I didn't have any room for era this is only a three or five day event so I couldn't wait for relevancy to kick in I needed to make sure I had those already in place so they kicked in automatically same with stop words half of our labs which I discovered the hard way have optimizing and transform in the lab name and so I actually moved those to what I call to the stop words so that if someone said how do I optimize itom operations we focused on the itom operations versus optimizing item operations versus optimizing the mobile experience so I made sure that I got rid of the um the terms we used a little bit redundantly in some of our session names and then of course boost in promotion rules so in my AI search profile I wanted to make sure that when users asked for when is Bill speaking I know exactly what they're asking for they're asking for Bill McDermott they're talking about Keynotes I want to make sure that those are surfaced first and then of course we're going to sort by the date today's dates are going to appear first future dates are going to appear underneath so here's an example of my search profile my k23 here you can see my sources I've got custom tables for venue information and for sessions I am using my knowledge base item or my knowledge base excuse me but I'm making it very specific to a knowledge base within my instance and I did have a couple of catalog items I did create a custom synonym um dictionary as well as stock words and in my synonyms and in my custom dictionaries I'm just going to go ahead and open this and just show you some examples of what we look for and if you think about it we're all familiar with servicenow so itsm seems pretty straightforward but in say like and this is how I think of large language models and natural language understanding what I thought of is that's not standard um vocabulary out in the world order management itol advanced work assignments so I made sure that those synonyms were all available so if someone said I'm looking for the session on AWA we knew they were talking about advanced work assignments and then of course my result Improvement rules okay so that was how I really improved search and I was able to take advantage of all the capabilities in AI search because when you think about it a lot of the questions follow the same format you know when someone's asking about a session and they might be asking for when is or where is the hackathon where can I find uh the keynote uh when is Bill presenting where is badge pickup is really no different than from where is um the store merchandise place what is AI Ops first we have to provide a synonym for AI Ops but then what is um for the product areas I want to make sure I direct to knowledge base so I made sure to put in some sample utterances and identify my source correctly and late to the um late to the game was our surprise guest in our chat bot which was any mention of check GPT or generative AI for that I actually created a virtual agent topic and the reason why I did that is because I wanted to centralize the sessions the speakers and the information just to a virtual agent topic that I could guide the user through and have it be very specific to the ask so I surface just those sessions in a what I call a unicorn virtual agent topic and so this resulted in just a better user experience so here's some examples of our evam consideration so if I asked when is Chuck tomasi presenting I could find um information about Chuck and see when he's organizing the Meetup now remember this is in a mobile app wholly dedicated to sessions and speakers so I just needed to point them in the right direction I wasn't the primary source of information so I just needed to give the user enough to say okay now I know what to look for in my agenda builder for my product pages what is customer experience management this is where my synonyms and AI search really helped so I could surface a knowledge-based FAQ with a link to our product pages and where a user was asking for venue information I could surface a card around say a meditation room a prayer room quiet space things like that now here's the thing when you have a mostly AI search within the conversation structure of the virtual agent I still have an opportunity excuse me up here that in my topic Discovery list I don't have any content to show right because I've only got one virtual agent topic and I've kind of hidden it and everything else is going into my content search so the challenge is and this goes to my third my principle of making sure that I am meeting all users according to their work habits this is where if we display show me everything I need to show something and topics for um just just having nothing show and show me everything really does create confusion for the user so I did enhance the custom greeting with tips I did decide that if I'm going to show me everything I'm going to show something and then I created topics just for search and there's a there's a wrinkle here I did create topics to to search specific subsets of my knowledge search profile and then I used one virtual agent topic that was nlu enabled for my generative AI topic and I hid that so that a user would just search for that and see it um but if they were looking at the show me everything they would not and then search my generic search that I just walked through picks up everything else and so what happens is something like this so if I open this experience oh so here I have a way to help users navigate through some search ideas so I can they can type in their requests here I'm helping with promoted topics but this guy up until I created some um some basic topics was empty and so what I did was I just said you know let's create a topic around let's make the maps which I'm displaying in my promoted topics that's going to be available on the list uh if I show let's go back to my show me everything now you notice check GPT is not on here but let's go ahead and let's look at venue and Event Center information so if you think about a user who is just kind of poking around they're like Okay um uh where is bagge pickup so this is what we're doing we're helping the user who navigates versus just typing it in at the um at the at the main level and so that's what I wanted to get to the show me everything list had to be a little bit more specific and user friendly but what I had to do here was I had to make it navigation only so I turned off nlu for these topics I enabled nlu for just this topic and this enabled me to balance my virtual agent nlu with my AI search content and that I thought was worth sharing because when I work with customers on the virtual agent we're thinking about nlu we're thinking about how users go through this conversation structure to determine if the virtual agent can help them self-solve but if I'm providing a lot of information there might be opportunity cities where I just want to get to content but that's not necessarily the most user-friendly approach in this case just providing a topic that is available from the list so I see it on the show me everything that someone can ask is their address code and get it answered I'm accommodating their work habits within my conversation structure okay and so I think we've gone through the demo and I did create just a quick wrap up right here so you could see it in action and so it's branded I moved it down to make it a little bit more mobile friendly so you wouldn't see the service portal login page taking up the whole screen so here where can I find a coffee station if you were at knowledge we had a few Demos in the hackathon around a coffee flow business process which were pretty neat so I'm answering a question around the venue coffee station and that I think I think we're almost done oh no no we're pretty much it I know I know we're over time and I want to be respectful of Time how are we doing on questions any questions that we need to bring up in the conversation the week yeah we do have one question sure uh somebody was asking you what's the the best or the most efficient way to to find your stop words so for example you mentioned that uh optimize and transform upwards like how do you go about finding finding those that's a great question and what's unfortunate is that I found that pretty quickly quickly grabbing a couple of session topics and uh I threw them in and because AI search surfaces multiple results at once that's how I saw that when I put in when it you know when is the optimize item operation session I got five results from AI search saying here's optimize item here's optimize mobile apps here's up and I was like oh well I need to get rid of the word optimize so it's just through testing I'm sure there's a more efficient way to do it in the real world but for for this if you think about it I had a limited um a limited data set and uh it's pretty fixed and finite for a very short amount of time so that's how I did it no that's that's great yeah it sounds like in that case uh in the context of of the knowledge event optimize and transform our words that we use a lot in the title of the session and they probably create noise in your search results so uh great way to to find those there through testing and make sure we we filter them out yeah Vanessa asked a really great question in the chat um how to make AI search um the virtual agent point to AI search just real quick um and Vanessa if you're still on the call thanks for staying with us but I do want to answer your question um when we have what we call a conversational experience where we configure the components of a conversation and it might be portal specific or language specific but you'll see two areas for two topics called AI search fallback and then fallback and when AI search is activated in your instance this AI search fallback Topic in your conversational experience becomes active so that's how you make the virtual agent point to AI search we provide an out of the box conversation topic that uses the AI search topic block and then in my case where I create created a custom content search profile I can just update that mapping so that it points to that Search application so it's it's really stored in a conversational experience on the virtual agent but that's how we point to AI search and I hope that answered your question that was in the chat link but I thought that was pretty important good question yeah absolutely yeah and uh if we had to I mean it was the session today was packed with best practices I hope everybody captured all of that both for the tradition and for AI search when we uh the 75 or 74 percent of our participants who said are using the video train agent I I would uh I would hope that most of them now see the value of using AI search as well in the virtual agent um I did answer another question that that um the AI search for HR knowledge I believe it's just a default dictionary I don't believe we make it specific to the workflows I think it's just one dictionary um I'm pretty sure that it's uh and it's pretty generic it's actually like when I created my product areas source so what I Tom itsm HR customer experience I actually created that whole Dictionary around those servicenow um acronyms you'd think that would be in our dictionary but it's not so I think the default dictionary is pretty basic I hope that answered your question Sean mcgathy uh let's see anybody else and if you have to drop thank you but I'm just going to stay on and make sure we've answered our questions okay uh Thiago AI search you can't do AI search on cmdb you can do it on requests and incidents and um this might be a good follow-up conversation on best practices for retention policy and filter conditions to make that a very meaningful search if you think about it what we're searching when we're surfacing these results to users might not be the millions of task records that we have in our in our tables but cmdb is not available for AI search just because of the complexity of the cmdb and Edson I think that's a brief overview about the search Matrix dashboard I think that I think the dashboards and search analytics might be a better uh topic because I think we've I think I've overstayed my welcome here yeah and we did a session two weeks ago when we talked a little bit about the search metrics and there was a session last year so I I would encourage you to look at the previous recordings we we covered uh user search metrics uh uh in the past but if that's the topic you're super interested about we'll make sure to have another session for that in the future and Thiago if you are still on the line Paul provided some additional Insight that if you still have contextual search so Zing you can use the cmdb there thank you Paul that is good information to know um and then Anonymous why do we need admin privileges to SharePoint to do AI search uh I think that's more around the integration um just that connection and credential to us from a secure user in um in assigning roles but I'm guessing whoever asked about admin privileges uh to connect to SharePoint I think that's mostly to build the connection um for usage it should just respect standard rules okay uh thanks everyone for hanging with me with the extra time thank you Trisha thank you everybody and uh we'll pause the recording when it's available and we'll see you in two weeks for our next AI Academy session thank you everybody have a great day thank you very much everyone