let's get started quick introduction before we dive into the subject of today so as always welcome to Academy as always I want to mention that we have additional resources so the academy is not the only resource we create we have an AI and intelligence Community Forum on the service now Community website it's an easy link to get there Snorks our team creates content we have experts creating content and that's a good place for you to ask your questions if you have any questions about one of our AI products this session today is recorded and posted on the now Community YouTube channel that's a YouTube channel with a lot of other resources as well and we recommend to check that out um in in those sessions we tend to cover what's available in the product but if you have any questions or if we mention some of the aspect of the products that are coming in our road map please keep in mind our safe har notice and uh regarding forward looking statements and always check back with your account team before making any purchasing decision if you are new to AI Academy what is that exactly it's a session where we bring content for you so we bring you fresh idas to get a better understanding and practical guidance of our AI product as I mentioned the session is recorded and then it's posted to be available publicly but if you are here with us today it's a great opportunity to ask your questions live and so for that we ask you to use the Q&A panel in the zoom interface to ask your questions and we'll get to those during the session our general agenda for the sessions we always have an in an overview then spend some time in an instance to show you how the product uh works or what to we covering today as part of the exercise and then we have some times for Q&A and today with me is Victor to talk to us about generative AI controller hello everyone thank you for coming Lo thank you for inviting me for to the AI academy uh for those of you who don't know me I'm Victor Chen as Lok mentioned I'm not B product manager like him uh for those of you who think I have a familiar name and voice it's because I run the virtual agent Academy which happens the day before yesterday uh so uh here I'm just a a guest star and on the AI Academy and looking forward to talking more so with that let's get started so today I wanted to introduce to you the generative AI controller so service now as you all know we have uh recently shipped uh several Solutions part of our now assist package or excuse me suite and part of that includes the controller and what this does is this allows you to embed generative AI in your service now workflows without any integration complexity so we in in part of our analysis Suite we have some things you may have heard um case summarization uh we have a Now assistant search uh we had resolutionist generation we have all these uh capabilities that uses generative AI but what if you want to create your own generative AI workflows you know or something a little custom well you can do that and that's you can do that with the with the controller and this allows you to achieve faster faster time to value for your generative AI development without having to deal with rest API having to deal with uh you know integration Hub although yeah uses that in the background but you don't have to worry about that it's it's all a low code uh integration with an with a generative AI provider whether it's Azure or open Ai and we're going to see how that looks like moving forward so it's simplified API and we have some outof boox capabilities we Shi to to you such as summarize generate content generic prompt and more we're going to use uh summarize in our uh exercise later and you can seamlessly embed this drag and drop into either flow designer virtual agent designer uh or scripting and for those of you um who may have watched the virtual agent academy yesterday we used VA designer yesterday we're going to use flow designer today uh next slide please so some things to out right uh first off to use a generative a controller you need a now assist skew all right that's either going to be now assist and Creator itsm hrsd or CSM uh that may you may have heard that being called uh you know Creator Plus or you know or Pro Plus or Enterprise Plus so you need that to use a controller it is only available for Vancouver patch 2 uh and above so you must be a Vancouver uh patch two to use this it connects to either Azure open AI or open Ai and you requires you to have an API key and subscription now Google Palm is coming soon so you have that to look forward to if you have neither of those uh but you know those are the two providers that's available at this very moment and finally there's no public access to now llm so we service now give you now llm to use again for case summarization for resolution notes generation and search but uh with and it and it does use the controller again in the background behind the scenes uh but end users and customers won't be able to access now llm through the controller again you can only use those uh providers I mentioned in that third bullet point okay uh next slide so we so we can go into the exercise but before that I wanted to pull the audience and L if you can do me a favor uh if you if you can run the poll I'm curious about all of you in terms of if you're EXP generative AI with service now or at all which provider are you using as your open AI open AI neither or perhaps something else and we'll give a 30 seconds to 45 seconds for that and uh while I do oh you know I spoke to the slides while I go through the exercise feel free to ask questions in the Q&A um L we uh and you know and Eliza can answer them or I can answer them at the at the end of the exercise and uh you we can go from there okay um we had 109 participants only 51 52 responses we can do more get those Mouse Clicks in yeah come on I'm sorry we're not raffling away a car but you know I would if I could um but so far from what I'm seeing uh slight Edge to aure open a ey but for for all of you uh it's none yet which is fine uh you know the thesis of service now has always been an extensible platform oh uh oh it's not multi select that's that's fair then just select the one you're you're most familiar with John uh but the thesis has always been our extensible platform so you know we really don't mind which one you use um and of course we're going to add more like Google like I mentioned but we are just curious okay I think we've got about oh we're almost yeah almost there but it seems like things have evened out L to um his natural uh conclusion but it seems like a slight Edge again to Azure open a followed by open a and then for all of you uh for most of you none yet which is fine but eventually you know you you you will or you could or you will great uh okay with that said let's get started into the exercise I'm going to start my screen share give me just one moment make sure this works you know what I'm going to flip my monitors give me one second and push this up there and then desktop 3 uh that's not desktop 3 it is a desktop hold on a sec so in the meantime just going to quickly answer one of the questions someone is asking us if all the Gen generative AI capabilities in service now require an outside controller and and the answer to that is no some of the out of the box bu oriented solution that are available in in the workspace and all of that they they ship as part of the outof thebox product and they tie to our internal model so you don't need to do any of that configuration today we're going to talk about how to expand from there yep so yeah these are for your own custom use cases you know perhaps you want use a little different or you have a different flavor uh of of what what we deliver or what you have probably in your own mind today we're going to we're going to cover how to use it very easy okay first you need an AI provider right Azure open AI or open Ai and you need to connect uh their API endpoint and API key to service now how do we do that we're going to go to connections and credentials if you've worked with other apis uh before uh you have seen this perhaps for example a dynamic translation or some other service and lo and behold we provide uh this out of the credential Alias out of box to you either Azure openi or openi let me back up even further let's install this plugin first so I'll keep that there for a moment but uh you should go to the service now store and uh install the generative AI controller again once you've purchased uh a now assist skew and the controller in the plugin you can see I've typed this in before H there's a couple updates for me okay I'll update it later um you know there's a generative AI controller Store app and then the store app has two dependencies the open AI generative AI spoke and the Azure generative AI spoke so you'll need these three at least for what we're going to do today okay and again when you install a now assis Store app these should be these three should should be included okay so once you've installed these store apps then we can go here the part of the store WS provides these two aliases out of box to you let's use Azure open AI as our example so I click into here and you notice we already have a connection here but if I didn't uh I would click new create new Hess related link and uh let me close my Microsoft teams just in case it's blown up in my other monitor monor uh so here uh you'll need your connection URL here and your API key here where do you get that you get that from your Azure portal so let me open mine up and you see I have an Azure openi resource called open-platform I know very creative and if you go to your keys and endpoints there's your endpoint just copy and paste that into here and for API key you just copy and paste into here and then you click create and it'll automatically create this connection for you if I click into here you'll see that my uh connection URL is my endpoint and if I click into here the credential there's my API key and then you're ready then you're ready to go you don't need to touch anything else all right so you created your connection and credential now let's go turn on some capabilities those outof boox capabilities I told you about so let's go back to the Navigator oh one more thing we should show I did not show this yesterday in the academy uh but if you miss it yesterday it's no big deal because you'll just get an error message telling you to do it all right there won't be any mystery but you do have to go to system properties and you do have to go to generative AI you can see I I've typed it in here before uh there is a generative AI provider that asks for what is the default provider and you just type in either Azure open AI or open AI I think if I click in here those are your choices you just put one of the two in in this case Azure open AI put it in there click save okay now we're ready to dig into the capabilities so you do have to set this up first so then we go to and you saw this briefly in the slide here in the exercise slide we go to this table CIS one extend capability don't worry if it's too long you just kind of remember once you type it in enough times or you just add add it to favorites right so you don't have to remember it okay so assis one extend capability and these are all the outof boox capabilities we provide to you some of these are actually service now usable only but the ones that we're going to go over the ones that you excuse me that you can use are um generate content and this allows you to generate uh articles based off information generic prompt which basically means you can ask whatever you want it you know you want and we used generic prompt yesterday in the virtual agent Academy um sentiment analysis which actually tells whether uh this information or this text what that you input is has either a negative neutral or positive sentiment and then summarize which is probably the one of the more familiar use cases for generative EI which basically says take an input and then summarize that input okay and this is the one we're going to use in flow designer today summarize so let's to to use this capability let's say I want to use this summarized capability I open this up I want to make sure that active is true and it should be true out of box I'll click here to change scope temporarily yeah make sure this is active you'll notice for those of you who are experienced with the uh open AI API that the parameters are here for you you have error error code uh provider response Etc right but as you can see this is all low code out of box provided to you so you don't have to you don't have to worry about that but we do want to go to the one extent definition configs related list and you'll notice that um these are all true which is good and one of them is default which is the Azure openai chat completion this this determines which model you use you can see you can either use Azure openai model uh or open AI llm uh GPT 4 chat completions GPT 3.5 plain completion gbd3 so uh you have six options right uh three per Prov three per provider I am going to use uh or make Default true to be the Azure GPT 3.5 large language model okay so once I've set that up I am now ready to use this in a flow or in virion designer so in my case we're going to use a flow we're going to use this in a flow designer so let's open that up close that have flow designer load so now let's create a work a use case I'm gon to oops I'm G to open this another thread and pull this pull this aside because I'm going to use this as my notes because I'm going to build this from scratch for you rather rather than saying oh we built it already it's all done um we're going to build a use we're going to build a use case where we when a in gets resolved to summarize its work notes okay and oh and then take those summary of those work notes and put them into the uh resolution notes field all right so that's an example of a use case now assist doesn't um ship that exact use case but maybe you want to do that for yourself let's use a let's create a flow for that so I'm going to go create new create a flow here now you know you could create a subflow and then put that into a larger you can do that as well but for me I'm just going to create a flow here I'm going to call it summarize work notes and I'm going to uh oops work notes and I'm going to add a two because I got I already got a one here or in behind the scenes so like I mentioned we want to you know when an instant is resolved summarize the work notes and then put that into a field all right so our trigger is going to be uh uh an incident that is resolved right so uh let's see a record that gets updated table incident I can open this up loading loading loading oh there oh H come on let me choose a table click click no not letting me use a choose a table I don't know why let me just uh try that again add a trigger oops record updated from there we go incident uh condition no yeah condition is when the state changes to resolved okay and we only run this once so that's our trigger okay so now let's go to our action what are we going to do when that happens so I'm going to click plus click action and I'm going to go down to the spoke the generative a controller spoke there it is and uh you see those are the four there's a bunch of others that are kind of in the you know background actions but you got generate content generic prompt sendon analysis and summarize I want to use summarize okay I I wonder if they have yeah basically this capability you summarize input text easy peasy summarize text to summarize now what do I want to summarize I want to summarize the incident work notes right so let me show you how to do it the easy way and then I'm going to show you why we might need to script this okay so if I were to summarize a text or I'm sorry a field that was already a string field like maybe long description or short description I would do it this way I would just grab use the data pill picker pick the record update trigger yep inant record and pick the field uh you know for example description right or work noes that's how you usually do it but there's a there's a little twist with a journal field like work notes because work notes is a journal field you can't just pull it in straight it's not going to it's it's not a string field therefore when you try to summarize it it's not going to summarize well because this the summarize action is is expecting text or a string so what do we do we got to convert that workn notes Journal field type into string how do we do that very easy three lines of code or three lines of script okay I'm no engineer if I can do it you can do it so we're going to click this script block here and we're going to declare we're just going to declare a variable called Journal field because I need to pull the the work notes field here and you in the comment it tells you you can you can you can grab stuff using fdor DOD so we're going to do that FD or. dat and as you can see as I type it autofills for me I just keep pressing enter and then dot trigger yeah I enter again dot Uh current right I'm just reading the comments here this I'm not doing this from knowledge or memory dot okay now the fields right now I'm going to find the work notes field so yeah I like I like how all of this is to me already maybe it's running out of letters here so I type in w yep there it is work notes okay so grabbing the grabbing that field and I want to do something to that field I want to convert that field to a string right so I'm going to declare another variable called Journal string all right this is this is going to be the journal field into in a string and I'm going to convert Journal field to get display value if though for those of you who've done scripting and now Glide or now before very familiar function to you and then lastly return that journal string that'll just basically returnal return the uh that uh string variable now which is a a stringified version of the journal field entry that's it all right done with that action basically summarize the work notes which is this string okay you can see we kind of borrowed from lines five and six in the comments there so that done and then finally what we want to do is update the incident record we you know once we summarize we're going to take that summarized uh text and plop it into the resolution that's field of that of that record okay so let's do that there I'm going to go back to uh let's see here uh where is my update record I'll just type it in here update there it is update record it must it might have been in core probably uh table incident again record I can just yeah the trigger record updated this basically This Record Yep this incident record and then which field do I want to put this into field let's go uh where's the resolution notes field and what I want to put in here use a data picker again this time I grab from the summarize action and you remember those parameters I told you about uh either you get response action status don't treat as error provider there's some there's some more here we want to pull the response because I just sent the work notes into the summarizeed action from the controller and it's going to Output output me the response I now want to put that response into the resolution resolution notes field of the of the ticket right all right I am done let's save this great now let's test in now I need an incident with um lots of workn notes right not lots of work notes some work notes and I don't know if I have one real quickly or I don't know if I have any so I'm going to look really quick I'm going to go to incidents I'll just put some in there how are we doing on time we're doing musle menos on time um so you can see I have some work notes in this one already and uh this one I basically copy and paste it from Wikipedia um you know this long text here but let's do this let's do this again so this is I don't know what this is about a plant I believe and I'm going to put this in another incident so put this in work notes I'm going say this oops this information is super helpful right looking forward to learning more oops more Etc you know I could type putting putting I could put in one or multiple work notes there we go so now I want to summarize these work notes so I'm going to just copy and paste here as you can see this is the um the out of box summarized tool but this is not what we built I'm going to test test that in there run it let's view the execution details success completed that's good and let's take a look at this ticket and you can see here in the resolution notes I didn't change it to resolve because this was a test here I didn't actually execute it you'll so you'll see that a summarization is put into the resolutionist field voila um you know some other wrinkles I could have done I could have added this you know after I create this flow I can I can put this I can attach this flow to a UI action to create my own summarize button up here right I could uh use this to create records I can use this to update records like what we've done here uh sentiment again sentiment analysis I you know sentiment analysis would have just returned a a negative here or or a positive here right in terms of the response so there's a lot of whole host of other use cases you can do this is just one example of a use case let me this resolve to put this back in there uh a lot of use cases in here and uh you know definitely look forward to for you to explore and experiment with the controller um as as you can see very low code we didn't script or code or call or study an API uh document uh whatsoever uh I look forward for you to all to explore uh the generative AI controller and with that that concludes the exercise and I can go back to these slides I don't think I have anything else on the slide it's just Q&A uh so with that yeah I'll to do some Q&A so um don't know if anyone wants to ask me questions or I'll just read them myself here um inline scripting bad practice this should be a dedicated flow action to do the conversion sure to each to each their own um and then Christina ask can you show what non-technical user experience what does a look and feel after the string is created oh I I believe way we just showed you oh or um you wanted the what the actual string looked like in that script um I don't know if it's possible to show but I think I can in the execution steps here uh summarize it might be in here Fields might be in here actually yeah provide or here text summarize this is what the raw text looks like in there and then Q&A looks like every all the questions being answered by L or Eliza thank you very much for that some good questions there and then Anonymous asks in layman terms can you show how this would look without the technical jargon how does it look to an agent work in a case well yeah so um this would be what the admin or the developer builds right to the agent who works a case and changes changes the um the state to resolved they wouldn't you know they they wouldn't uh have do anything they change resolved and then after a couple seconds we can even um let's see how do we do on time we can't do it we can't do it now but if I wanted to right I would not run to I would activate this and then I would give the agent a h up hey we have this new feature where every time you change a state to resolved it'll automatically update the resolution notes for you now uh when you change this to resolved usually the resolution know a required field you can just put something in like you know 2 be up 2B updated right dot dot dot and then when you click update it'll catch the change State change as a trigger and automatically do this you know up input this for you and the agent doesn't do anything they just see that the ticket being resolved automatically so again very straightforward go back to the slide here full screen it uh Jose asked well go ahead sorry now I just want to say I think it's amazing it we went super fast but it's amazing to realize that in 20 minute we basically built um very easily something that integrates with a very Advanced model and we able to perform a task that otherwise would have been impossible a few years ago so it's it's pretty it's pretty impressive it it was fast but I think we can take time to just realize that it's pretty pretty impressive and watch the video you can watch the recording again and again at 75 speed if you want to uh but it's a kind of the beauty of I mean I'm not a flow designer expert but I kind of figured it out because flow designer also is just very easy to use uh you know VA designer for uh the other for the other flow we did yesterday also pretty straightforward so um yeah that's that's the power of low code tools in service now um we can how are we doing on timer do we have time for other questions yeah we have uh a few more minutes if you want okay yeah um in the chat Mel asked are we going to receive these instructions um yeah the recording is going on YouTube you can just follow along or you know add your own add your own twist the real you know instruction if anything the new twist is using that summarized action or one of the other or one of the other generative actions you send it an input you receive an output in that response parameter and from there you know the r is your oyster in terms of workflows you want to use for uh in in in flow designer or or scripting which we didn't show but that's documented or VA virtual agent designer which we covered yesterday uh Eric ask can you specify the format of summarization string just text uh and then uh VJ asks is there a fine-tune llm related to itsm test that will be abl for use within service now that's what now llm is so you can't you can't use that in the controller yourself a service now uses that for the now assist solution Suite so um if you want to look if you want to look for uh now now assis for itsm right uh that's use that uses our fine-tune model uh for uh incident summarization for resolution notes generation uh and search Etc uh but for the controller you can just use uh the two providers I mentioned earlier in the slides uh Azure which I used or open or open AI great so just going back on the question about the format uh if somebody wanted a shorter summary a longer summary or summary in in bullet points then would they have to use like the generic prompt instead uh probably I've I've seen that question before I haven't tried it myself but yeah that could probably work yeah especially GPT 4 um for those of you familiar GPT 4 allows for you know richer form adding the output it's all it's all in the prompt right and generic prompt you can basically tell to do whatever U you know give me a recipe write me a PO or summarize with XYZ you know prompts and and and it should be able to provide you with uh with with the output that you are looking for at least close to and then Susan asks uh this was a third partyy llm which is true if you wanted to build our own customization with the service now LM how do we do that uh as I as I mentioned you're not able to uh the now llm is our internal llm that we use for our now assist solution so um you know as of today uh you're not able to use the now llm uh with a controller just your third party llm subscription all right I think that's a good time to wrap up today thank you Victor for the very insightful presentation I hope everybody got a lot out of that all right thank you everyone