all right everyone let's go 
ahead and get started with   today's conversational interfaces 
Academy thank you all for joining us as always please be sure to visit our 
academy our Academy page on our virtual   agent nlu Community Forum there you'll see all 
our past recordings of our Academy you'll see   what's coming up and of course our community 
Forum as well as all the information you need   you can go there look up articles ask questions 
give answers all of your experts just by virtue of   attendance Academy and if you're watching this on 
YouTube and you like what you're seeing go ahead   and click that like And subscribe button that way 
you're notified every time a recording is posted   for example this session here will be recorded 
so feel free to share this invite to all your   colleagues all your friends family members your 
pets maybe but if not they they can certainly   watch it afterwards once it's once it's posted on 
YouTube recording usually comes up later in the week and today's an exciting Topic at least 
from what I hear you know every time I you   know that the word generative AI have become 
a pavlovian response in Tech Community and   today we're going to talk about using the 
generative AI controller with the virtual agent so a brief agenda and I I usually don't 
have an agenda because it's very simple but   today we're going to do a bit of a breakdown 
so that's why I have it up here we're going   to first talk about the generative a controller 
overview what it is what it does we're going to   do an exercise how to turn on the generative 
AI controller how to activate it and then how   to use it in Virtual agent we're actually build 
a conversation with it then we'll talk a little   bit about road map in terms of what's next 
for now assist and virtual agent for those of   you who don't know now assist is the um term for 
service M generative AI Solutions and then we'll   wrap with Q&A it's just me today so apologies 
ahead of time if I'm a bit delayed in the in   the in the responses for Q&A I don't have an 
assistant today but we will make it work all   right so what is the generative a controller so 
the controller allows you to embed generative AI   workflows in your service now platform without 
any um integration complexity in the past if   you wanted to integrate large language model 
in the service now platform you'd be calling   rest apis you'd be doing some scripting maybe 
blow the flow you would incur integration Hub   transactions and it's and you know the the 
request response Isn't So performant Etc   well all of that is is gone with the controller 
you can do all this now without any integration   complexity you can achieve faster time to Value 
it's all low code there's no coding no scripting   none of that to create value and to accelerate 
your generative AI development in service now   so as I mentioned it's a low code integration 
So today we're going to when we go over this   it's all it's there there'll be little to no 
code and it's a simplified API and we provide   a lot of outof boox capabilities for you things 
capabilities like summarize like generate content   generic prompt and more we'll go over those 
in a bit later and then you can seamlessly   incorporate this into design experiences such as 
flow designer virtual agent designer which is what   we're going to go over today and scripting 
if you if you choose to but you don't have to so you know some general quick tips and 
FAQ before we get started first off the the   generative a controller requires a now assist 
skew all right so talk to your account rep or   talk to your service now platform owner if you 
want to use a controller but we have some SKS   available the controllers available ailable is 
is available in all of them once you purchase   the skew you can download the controller okay the 
controller is only available for Vancouver patch 2   and above so you must be on a Vancouver patch two 
instance to use controller and then moving forward   of course you'll be able to use it in order to 
use a controller you do need an Azure open AI   or open AI subscription and that subscription 
has a key you'll need the API key I'll show   you later on where to put it and then coming in 
the future you will we're going to enable micro   or I'm sorry Google Palm so that's cool and then 
just to note there's no public access to now llm   so you may have heard of us having now llm which 
is our own service now llm yes it does also use   controller in the background but that's not um 
for customers consumption it's for um um it's   for interal development tools so just keep that in 
mind mind Josh quickly asks is there a tentative   timeline for November for Google Palm yes there 
is later this month and I'll have the the forward   looking statements Safe Harbor later on but yes 
it's it should be tailored for later this month   so let's jump right into let's just jump right 
into our demo exercise so there's going to be   two tables you need to be aware of now I know 
these are long table names and stuff so that's   why excuse me I leave it on here for you take to 
copy paste or to write it down one is called CS   one extend capability and that's going to be where 
you enable or turn on the controller capabilities   and the other is CIS one extend Builder config and 
that's where you turn on the capabilities for the   builders whether that's Flow in this particular 
example virtual agent so let's Jump Right In   I'm going to pause or I'm going to stop my screen 
share I'm going to switch to my instance and we'll   get right on it but the first thing we're going to 
show is the store this is where you you know once   you purchased analysis skew and um you know that 
does entail an additional cost or you know as all   SKS do once you download this once you download 
the analysis skes you can download or install any   of these plugins or these store apps technically 
plugins and they they all have a controller once   you download and install it you go to the 
plugin manager you go into your um your your   plugin manager you should see amongst your now 
assist Store app installed also the generate AI   controller installed the controller is dependent 
on and will include two other um apps one is the   open AI spoke for your open AI subscription the 
other is a Azure open AI spoke for your Azure   Openai subscription okay the other thing you're 
going to need is um as I mentioned a subscription   for example Microsoft Azure we're we're going to 
use that as an example moving forward so here if   you go to your Azure portal I'm going to there 
we go sneak that in there um You probably have   an open AI Azure openAI resource and when you open 
that you know you have your information here you   go to keys and endpoint and you'll have your 
endpoint and you have your key right so you're   going to need this later on so let's go back to 
the instance and let's try start configuring all   right so once you got all this installed we're 
going to go to that table I mentioned which was   CIS one extend capability don't worry you type 
this in enough you memorize it like I do now these   are all the capabilities that are included for you 
via the controller now some of these your mileage   may vary they may they may work more than others 
I don't have them all documented I haven't tried   all of these but the ones I have tried and do work 
oh and some of these are also internal meaning you   know you're not meant to use them they're service 
now built don't worry about it it's all given to   you automatically right um the ones that do work 
are generate content and this basically generates   content such as knowledge articles or articles in 
general right of generic prompt and we're going to   use this later on generic prompt just basically 
means ask the llm anything so if you imagine   like Chad gbt we can ask it to you know write a 
poem or give you a recipe basically that's what   generic prompt is ask it anything give it anything 
or answer anything uh sentiment analysis we you   can use that one you can input a paragraph or a 
sentence and it will output sentiment analysis   negative if it's a negative sentiment neutral or 
positive sentiment and then summarize as well uh   summarize basically you give it a Big Blob of 
text and it'll summarize it for you okay let's   see if there's any more nope that's it everything 
else is either your miles may vary or it's meant   for service now internal consumption but we're 
going to go over those four okay so let's look   at summarize that's the well no I'm sorry let's 
look at sentiment analysis that's the first one   we're going to use that's the first one of the 
first ones we're going to use in our virtual   agent so clicking susum analysis you may have to 
switch Scopes to open it up a bit let's make sure   this active flag is true which should be automatic 
right you'll see that there are some attributes   here now for those of you who are familiar with 
the um Azure openai API these parameters will   look familiar to you right error error code 
provider response status Etc right again this   is all low code so rather than reading the API 
documentation these are here for you but then   what we want to go to is the definition configs 
um related list and you want to make sure that one   of these definitions is defaulted to true all of 
them are active to true or they should be so you   know check that but these this determines which 
model the controller is going to use and you see   we give you six right open AI Azure open AI cheat 
completion GPT 4 just plain opena completion this   one's gpt3 and then what I have available and I 
can I can open this up here just to show you more detail right make sure this is true make 
sure this default is true aure open a check   completion this is GPT 3.5 okay I recommend you 
use GPT 3.5 or gp4 they're they're faster and I   believe GPT 3.5 is cheaper but it's ultimate up 
to you right this is this is what the going to   determine what model you use for your um input 
output okay so we make sure that's true now we   got to connect your azure openai subscription 
to the instance how do we do that we go to   the connections and credentials Alias page 
now for any of you who have connected with   other you know apis this page is familiar 
to you so I type in connections there it   is connections and credentials credential 
aliases and yes the first one Azure open AI now I see I already got one here but if I 
didn't have it this is blank you simply click   create new connection credential and remember 
I showed you earlier the endpoint you copy and   paste your endpoint into here right and then 
you copy and paste your API key you know you   can click to copy the keyboard and you paste it 
in here you you click create and you're done and   that and then this record will appear for you I 
can show you what this record looks like there's   connection URL and then the credential will be in 
this record there's the AP there's the API key and   you'll be done with that of course if you were 
to use open AI they also provide you an endpoint   and API key same process I believe there is 
an outof boox open AI recording credentials   in Alias there it is same concept okay so once 
you've connected that you've done you've you know   you've activated at least one of the capabilities 
right summarize if I go to history if I go back to or not summarize I'm sorry I keep saying 
summarize sentiment analysis um I also did   it for a generic prompt and because we're going 
to use that in a moment and you'll be good to go   you'll be able to use it in um any of the builders 
right flow designer virtual agent Etc let's use   it for virtual agent because this is after all 
a virtual agent Academy so I'm going to go to   that other table remember the one I mentioned here 
this one here so let's go to there Sis one extend   Builder config if you haven't memorized it that's 
okay I put it in my one note to you know so I can   copy and paste it later and you'll see there are 
a bunch of Builders here um the one we want to   look at is virtual agent designer okay let's just 
make sure active is true right all these should   be defaulted but sometimes you never know right 
there should be a sync and active and you'll see   I put it a bunch I put a bunch in here in here 
already I'm going to delete this code is this   one this one isn't relevant but maybe I'll do it 
again just to show you how what it looks like but   basically this will be blank you see I've already 
included a couple in here but I'm going to click   on new and I'm going to add one that's already 
been added but I won't click SM I'll just show   you what you do so you add capability and hope 
this popup window appears for you I'm sharing   desktop yeah so it should and which capability 
do you want to add to the virtual agent right we   want to add sentiment analysis so I scroll down 
there there it is sentiment analysis I want to   make sure a execution mode is async always async 
and then if you wanted a different icon in the   virtual designer you can click to add here 
it'll it'll bring up your file explorer you   can add an icon like you know happy face because 
of sentiment analysis or maybe you want the open   a logo whatever right you can you can do that 
you click submit and it'll show up here send of   analysis I did the same thing here for generic 
prompt def default doesn't matter so much here   but you definitely want to make sure active is 
true for all of these right so I did I did the   same thing for generic prompt I'll just show you 
as an example there it is generic prompt a sync   as always uh description if you want to doesn't 
matter and an icon if you want it to add that as   well all right so once you've done that these two 
capabilities these two generic I'm sorry generic   prompt sentiment analysis these two generative 
AI capabilities are now available for you to   use in Virtual agent let's take a virtual agent 
designer let's take a look at that so let's go to designer and I see there are no questions so 
that's great this either means I'm doing a great   job or it means that I've completely overwhelmed 
you but so if it's one or the other give a shout   out in the chat if you have questions feel 
free to start piping them in Q&A we're at   we're at the home ining at home stretch of of our 
exercise so bear with me here but here's where we   everything comes together I want to create a new 
topic uh let's call it generative AI test call a create and you'll notice uh in the left hand side 
of little components if I scroll down there they   are generic prompt and sentiment analysis okay 
so let's do this excuse me let's do this case   here let's ask the user how they're feeling in 
the virtual agent we'll analyze a sentiment and   if it's positive or neutral we'll say you know 
yay if it's negative we'll ask them to expound   and how's how how does that sound okay and 
we'll have them expound to say hey how else   can I help you and we'll pass through a generic 
prompt there okay so let's do that very quick   first we're going to ask how they're feeling 
how are you feeling and the prompt is how are   you feeling okay and then we're going to drag 
in our sentiment analysis it's a topic block   technically speaking but you'll notice here your 
outputs remember those those parameters in the   capability right and there's your utterance we're 
going to say we're going to call this this is a   sentiment node utterance what are we passing in 
as the input we're going to pass in whatever they   typed in here right so we're going to use our data 
pill input variables how are you feeling okay now   what this returns is either um a NE literally 
the words negative neutral or positive okay   so let's create a let's add a decision tree 
because we're going to do different things   depending on whether you know what our output 
you know is right we'll have one for positive neutral and we'll have one for negative Okay negative that's if the response okay 
so status is usually just you know okay right so   it's a 2011 301 something like that right but 
responsible what you want right provider is   it's it can tell you whether it's h it's coming 
from open AI or Azure open AI that's good but we   want response if response is negative we'll have 
we'll have it do something and then if response is positive or response is neutral we'll have to do something else so 
positive neutral we'll just say we'll just say   okay good to hear okay that's great to here talk 
to you soon and we'll end the conversation there   right uh but what if it's negative what do we do 
right so let's um ask them you know what would you   like Sor of here what would you like right uh 
what do you want so three two here how how how   can I help you all right and and you know a more 
work rated use case here is maybe for AI search   or some search fallback let's say you search your 
results results don't turn up anything okay now   you want to search like the llm now you want to 
do like a chat GPT ask anything sort of deal right   if it if the sentiment say is negative and like 
the search results did not help me then you can   do something like this right you can ask okay what 
do you want uh to to know right and we'll drag in   the generic prompt topic block remember same thing 
got your parameters you got your prompt right this   is we'll call this you know ask anything and the 
prompt input will be uh there it is what do you want and then we show the output we're going to 
display the output of that ask anything okay so   we're going to call this this is the generic 
output and the response and this where gets a   little tricky because now you got two um topic 
plus you've got sentiment analysis you've got   generic prompt so your input variables you're 
going to see error error two one two provider   one two response one two right so you want two 
obviously because this is the second one you   you pulled in there response or you know this is 
what and this this what this will do this will   show you the output of the um generic prompt but 
let's do a little bit fancy we want to make sure   we want to we want to um you know site a source 
right so we want to say that this is provided by   space provider two all right and then enter and 
then your response and what this is basically to   do if I ask the generic prompt like you know 
write me a poem about you know about flowers   it's going to say provided by aure open the eye 
and then a poem about flowers okay so I think   we've got everything here again if you have any 
questions on what I did feel free to put it in the Q&A all right we've published 
this now usually you want to   test it before we publish it but we're 
kind of pressed for time so let's just   publish it test active topics hopefully 
you guys yep the popup shows up on my desktop and we want the generative AI test how are you feeling let's do you 
know I I feel great yay and it'll tell   me you know good job thank you or what was 
it great to hear yeah talk to you soon and   if you go to variables you'll see that 
provider was Azure open Ai and response   was positive that's good success status 
of course success oh good doesn't just   give you codes it gives you words that's good 
let's do the opp let's do the other uh version right show me everything gener VI test again I feel bad not getting help 
boohoo I'm really laying it on thick um and you'll notice hopefully this gets refresh 
yeah response is negative now we're in the other   Branch started here how can I help so now I'm like 
well I like breakfast I like scrambled eggs how do   I make scrambled eggs this is basically again 
you can ask at anything pass it any utterance   almost like chat gbt basically and you'll 
see here your provider two is azure opena   and your response is how to make scrambled 
eggs provided by a open a ingredients oh wow   milk cheese that's that's really fancy scrambled 
eggs and there you have it how to make your users   feel better of course your users would probably 
want to search something more relevant to their   work if their search results didn't help them in 
our work rated use case but yeah voila so that's   how you and again no scripting no coding No 
Rest API to or parameters or or API Ducks I   basically you know everything was given to you 
in the controller you drag and drop you turn   on some default and active Flags to true and off 
you go all right great now let's go back to the   slides we're going to cover some uh some other 
things really quickly and then we're going to do Q&A so what's next for Now assistant virtual 
right so what's coming in November January and   even in and then of course in March which is our 
Washington release so now assistant virtual agent   just came out in Vancouver in September you 
may have seen the press release for that and   that's essentially enhanced uh AI search in the 
virtual agent right so rather than getting a list   of links and virtual agent uh rather than getting 
a gen result you can get a you know a generative   relevant context result uh in the virtual agent 
okay in November we're going to enable multi-turn   with knowledge and service catalog in other words 
right rather than just getting one asking one   question and getting one response you can start 
conversing with that knowledge article asking   multiple questions followup questions right same 
thing with service catalog right we'll be able   to you know ask okay you know naturally what do 
you want get into the details of what you want   Etc we're going to improve the uh setup guidance 
right so we understand that this will entail some   more configuration Etc we don't like you know we 
don't like um doing to configuration we're going   to try to simplify that as much as possible it's 
going to involve the conversational interfaces   homepage right that uh that low code page with 
all the configurations we're going to make that   uh easier to incorporate some nist capabilities 
um we're going to incorporate virtual agent in   AI search so even um you know yes there's AI 
search in Virtual agent but for AI search in   our portals we're going to incorporate virtual 
agent we're going to going to do the reverse do   virt incorporate virtual agent topics into your 
AI search in Portal right so if I ask a question   and virtual agent may be able to help you better 
we can Surface those topics and then finally and   this is perhaps the most um General but I'm going 
to just put it out there we're going to shift from   nlu to large language models so nlu is not going 
to go away and if you have nlu topics that's good   because um we're going to provide some method 
for you to say okay you have nlu topics how do   we make them into llm topics right so um that's 
going to come a little later than the first three   bullet points uh but I wanted to raise that of of 
course Safe Harbor applies uh to forward looking   statements but just some things to be aware 
of all right but wait there's more so today we   talked about the controller with virtual agent 
tomorrow I'm going to be guest starring in the   AI account academy uh tomorrow morning Pacific I 
believe the same time maybe a little earlier and   I'm going to talk about the controller again but 
I'm going to use a flow designer example okay so   if you're registered for the AI Academy that's 
great if not definitely check it out I think   if you go to the community Forum go to that go to 
our Ai and intelligence Forum you can register for   the AI Academy there I believe the registration 
page that one that has all the um all theems AI   Academy is in there too so yeah check it out all 
right by now I'm sure you have questions I'm going   to try to go through them as quickly as possible 
apologies if we stay a little later just want to   get through these questions so let's Jump Right 
In um yeah Eric asks will gen replace nlu it won   maybe in the long term right um but nlu will still 
be around customers still use nlu people they they   like it or they they have the outof boox models 
we're not going to get rid of that immediately but   we're going to offer a way like I mentioned hey if 
you want to if you're on llu and you want to shift   to llm we're going to give you a way to do that 
so this is not a replacement it's going to be uh   you know it's going to be an option for customers 
to experiment maybe L maybe the llm uh does some   weird things to uh your out ofox topics that 
uh you just want to keep in on you first right   while you do some tuning or do some configuring 
so uh it's not a replacement it's just a shift uh   Marius asks is it possible to activate AI search 
plugin for pdis um I don't believe so because   AI search does require uh a machine learning or 
AI connection and those don't do very well with   PDI so I'm going to lean towards a no on that 
if there if there will be a shift to llm does   that mean the control customers will need to buy 
the AI controller eventually well you see that's   a good point if you don't buy the controller if 
you don't buy a purchase now skew then you're   going to stay on NL and that's and that's a fine 
thing so if you want to shift to llms yes it will   it will involve analysis skew if you don't that's 
fine you can keep using nlu again it's not going anywhere are we ever going to be able to train 
an llm on our internal knowledge base and have   VA questions well yes not train an llm at least 
um you know list to some difference there but   you will be able to fine-tune your llm with your 
non space articles and we actually do that now   so we actually use uh ra R A or retrieval 
augmented generation today to augment the   LR uh your llm to answer questions basically 
your knowledge based articles and soon your   service catalog as well so train isn't isn't the 
right word more like augment okay uh will there   be be a capability that allows passing data 
to Azure and have the prompt ask opening has   something about that specific data um not not 
an outof box capability that does involve you   know using external source which isn't available 
today so right now your data source is just just   within the service now instance and that just to 
make things you know easier reduce integration   complexity Etc we may enable external sources 
in the future uh I don't have visibility into   that road map item um but it's a good question 
but right now it's just for your uh service now information okay uh let me going to check the chat 
to make sure I'm good everything was good yeah we   me we mentioned the SKS there um I'm not sure uh 
Travis asks are there plans to integrate this into   the service now platform proper rather than just 
a customer support portal not sure what we mean   by that uh but we actually do have uh it's funny 
we do have uh we do use some generative uh Tools   in our support portal for in for our employees 
uh but nothing that' be customer facing uh just   a little topic is there a way just uh justinia 
I hope I pronounce that correctly uh is there   a way to implement pasting images to the chat 
directly from the clipboard I I've heard that   request before um agents can do it into their 
chat but users can't yet um I understand that's   like a a UI Gap but uh unfortunately not yet and 
then last question tago uh sorry if I missed the   presentation is there a way any way to capture 
the confidence level of sentiment detected yeah   so it's it's it's in the it's in it was in the 
response so if you saw that variable uh previously   negative neutral positive that's an invariable 
which you can use to you know to whatever you   want to do whether some sort of logic or some 
workflow like I did positive you know do one   thing negative do another so yes that's there all 
right oh one more question and then we'll call it   a wrap for the virtual agents is there a special 
format needed for knowledge articles so that can   view the content well if if you want to display 
a knowledge article in Virtual agent um no there   shouldn't there's no special format needed uh it 
can display anything basically anything text in   bullets if you have images in there I haven't 
seen that uh before but I'm sure it works   or it worked to some degree but certainly 
text and bullets you know that that that's fine okay so that's all for questions again thank 
you all for joining me uh I will see you next   time on the virtual agent or the conversational 
interfaces Academy and I'll talk to you all soon bye