thanks everyone for joining um as i said people just joined dell cheeseman here now intelligence group senior advisory solution consultant my responsibilities in the organization in the pre-sales world is i'm the specialist in the in the now intelligence group and we're responsible for performance analytics virtual agent predictive intelligence uh process optimization and the ai search so what we want to focus on here today is actually the natural language itself nothing to do well something to do with the virtual agent but we want to sort of focus primarily on the nlu because i think there's a lot of documentation out there in the virtual agent and there's a lot of documentation out there in relation to nlu but some people are more focused on the va getting that front end piece working but they sort of forget that there's sort of work to be done in the back end in the nlu so i just wanted to sort of focus on that so there is a lot of material out there on you know best practice of the nlu and multi-language support in the nlu this is more of a higher level introduction to the nlu but we will be going under the hood so i hope you enjoy this so first slide we always place this up this is our safe harbor slide and this is just in the interest of protecting yourselves and not those that you know there may be comments uh uh statements made in this uh in this session where we could mention forward-looking statements into the future to forward uh releases so we always sort of put this up to say please don't plan your deployment based on our roadmap as things can change and slip so with that moving through uh a bit of information about yourself myself i need to update this slide i need to add another year on so uh my name's dale as i said um come from a background of working in a service desk from the bottom and i've worked all the way through so i've worked at siemens i've went to astrazeneca i've went to ibm i've worked at match which is fifa and i've done various roles from first line all the way through to ops director and then i joined in the consulting world i was with some of my competitors as delivery consultant until i found my place at the servicenow family and i've been here now four years on the 1st of july so really happy here and great company great product so my day-to-day job as i said is in the pre-sale with support in the uh sc's and the account execs pocs demos but more in-depth on the technologies there on the left-hand side also responsible for the chain of game workshops which has been a global success workshops which helps you understand uh performance analytics virtual age and predictive intelligence and also helping our customers identify automation potentials in the clustering analysis workshop so this is what i do during my day and i always love doing these for the team because it's a great way of putting our messaging out there and for us to sort of get the bigger message out there about some of our products so this is a live on service now join us for future webinars and meetups on upgrades itsm hrsd item and much much more learn more using the link in the chat so that's the the marketing stuff that way so uh please tell me um obviously i don't mind people asking questions but we sort of like to leave the questions to the end because i factor in time for that but if you do have a burning question you know please use the q a feature to ask the question i'll try and answer most of them during the session if not i'll follow up on them when you come off mute please feel free to introduce yourself when asking a question we want to know we'd love to know who you are where you're from what it is you're doing uh the session will be recorded and shared on the servicenow community forum after the session and after the session ends you'll be prompted out to fill out a short survey this is where you we appreciate your feedback where we can improve you know maybe you can enhance and and and basically grow this particular webinar this is the first time we've done this particular webinar so hopefully we've got it right but if not please feel free to feedback now as i said my day-to-day role is i look and get involved with a lot of customers who are starting their journey and their journey into what we refer to as the virtual agent nlu which is actually what we now refer to as the hyper automation world and some of the challenges that are faced with a lot of customers is well where do i start with the nlu or how do i make my va more intelligent than just a standard and i don't like to use the word chatbot because chat is indicative to a customer that it's a bot that's grasped loads of questions i just wanted to be smarter than that i wanted to be able to skip questions and actually figure things out so the nlu is actually behind the scenes doing things for you in that world how do i set this thing up you know getting started with the nlu you know it can be kind of overwhelming but actually i'm hopeful i'm going to give you a really introduction level of this understanding where to get going and then grow your deployment after that then you've got the best practice you know on the nlu what should i do to keep my nlu clean again there are pitfalls and mistakes that customers fall into and hopefully i'm going to sort of outline some of these sort of things to watch out for and as i said there are other videos and contents available where we talk about you know refining fine-tuning your nlu but there's some new features that have now released in san diego and further uh plug-ins that have released that are actually helped doing things like batch testing and and collision testing so we're going to go through some of this as well so we've got a lot to get through now i'm going to put this slide up because this as i've mentioned before is the hyper automation world and for anybody who hasn't heard of hyper automation you're going to start hearing this more and more as we go through the year hyper automation is our strategy our four pillars of success in the discover automate apply ai and optimize world where in each of those pillars we look at the different parts of the now intelligence products where we start to understand a customer wanting to mine data to find their process issues looking at utilizing our automation hub to where we can automate procedures that historically we could only do if there was an api now we can do by using the automation rpa hub to be able to do that and grow that faster looking at the virtual agent and the predictive intelligence you're in this webinar is all aimed in the ai they apply ai area there's a whole host of things that we've got and we've got a whole showcase coming up and recorded content and live webinars launching this month on hyper automation so just to let you know if you start to hear the words hyper automation the service now will this is where we're trying to automate as much as we can as fast as we can for you to get the benefits in that okay so so when we talk about the bot we're going to talk about the high level the bot straight away because ultimately when you're looking at virtual agent there's many ways you can interact with the bot and on the left hand side you can see text message whatsapp via the mobile you can use slack teams workplace messenger that's great but understanding the flow all this great stuff you've seen the middle here you know it can the virtual agent can understand the user profile can understand outages cases asset knowledge base internal uh internal and external integration to augment data routing to the the best agent brilliant but how does the customer get there so when they open up microsoft teams for example and they say i need a new laptop okay how does the bot know what the conversation is so you know many releases ago pre-new york we we had a bot that launched in london and the bot was what's called a keyword search bar in other words all the bot knew was keywords to trigger a conversation and that's relevant in this flow here because if we talk about prior to new york before our nlu came on board and what you've got to understand is a conversation in va has a start and an end okay when the virtual age when the customer types and utterance in it basically says i need a new laptop for example or i need to update my profile that will use what's called the topic discovery to find the most relevant conversation and that is build up built up from having an nlu model with intense and then utterances contained within the intents but then if we look at this dialog flow and this exists even today when a conversation starts it has a flow of points from start to end and it has to follow that flow it cannot deviate so if for example this was talking about i want to update my email address so customers comes along and says i want to update my email okay it will start the conversation off and it will basically say okay what do you want to update well i want to update my email okay what is the new email the new email is this okay we're done so this would be driven by a keyword search so when the nlu come along in new york and we've enhanced that so much now is if a customer now says i want to update my email to johnson.com if we think about this flow when the first question was what do you want to update do you want to update your it could say do you want to update your email address your phone number your secondary email address you would basically say okay well i don't need to ask that anymore because you've actually told me what you want to update so this is where it could start to skip the question so i don't need to ask that and then the next one is basically and i want to update my email to johnson.com well here the process could have been what's the new email address well guess what i don't have to ask that anymore so i can skip that so it's understanding that the nlu will find the relevant conversation but as well as that it will also help you make the conversation run smoother and basically be able to skip questions and the old adage one in here is i always joke about this but when you're creating a ticket sometimes a virtual agent will say how urgent is the ticket well guess what people are going to put i'm going to give you the options of high medium or low well you might as well just default that to high because high percentage of customers are just going to hit high because it's important to them but if they said something in the utterance like my machine is on fire and i need a replacement today based on the words let's say fire and today it might say well i don't need to ask that question because the machine's on fire so i don't need to ask is it high medium or low because it's high so i can skip that question so that's what is called intent recognition so nlu drives the topic discovery to find the conversation and it also basically defines the intent so if you look at these short clips while i'm talking through this is to speed this webinar up is that if you see here i want to update uh it says here my uh profile okay that's all i'm going to type topic discovery will initiate and come back and find the relevant conversation so here you go i want to update my email so subscribers found it and we're stepping through and the customers having to type in i want to update my email i'm having to update my i'm having to type in what my email address is and then i'll arrive at that conversation now let's run the same conversation again with a customer with an nlu enabled utterances intent recognition so here when we run it here watch what the customer says i want to update my email to a nebeal at sn.com so the first prompted originally was what do you want to update well now as you can see here i've updated your email so very simple just to give you a high level understanding is that based off the same conversation being used the intent recognition understands the concept of the word email so i don't need to ask what you want to update and i and also i put in my email address therefore i don't have to ask the customer so that's what this is all about and on top of that you've also got to think about conversation switching now conversation switching has been around for a while but the nlu is there as well the topic discovery is doing this so here i want to update my profile and again this is a crazy scenario here i'm putting okay actually i don't want to update my profile right now sorry i want to order a laptop so totally different conversation so the topic discovery realizes that actually i have a better conversation based off what that customers just said so i've started one conversation and realized actually there's a better conversation and in then in enhancements we've made since we've actually also introduced the point of before i switch a conversation do you want me to ask the customer am i okay to switch and it's just an option but this is the high level understanding of what these are so hopefully that makes sense so think about the nlu and the topic designers so you've got your topic design on the right hand side where you're building your conversations the va and on the left hand side you've actually got what's called the nlu where you're building your intents and your utterances and in the san diego release we've actually made it even easier for everybody because we've actually introduced the guided workflows so we've actually streamlined our model creation process we've introduced some new functionalities and i think for me what's what's been a big jump in san diego as well is the ability to import data from a csv i have many customers who said you know what i've already got an nlu and i want to bring it into the platform so i've got you know 10 000 utterances across multiple intents in my other instance well actually i can import that data in now so i can i'm not losing anything i already have we're also using intent for pre-built models to create new models so we've enhanced that with the model builder we've also brought in the way of managing the phases so to step you through you know how you build your model how you test your model how you publish the model and also as well as that here you can actually see there is a conflict review and a batch testing are integrated as well so we've enhanced that in the san diego release also and the nlu workbench again you know allows a person to model an author and provide feedback on the intent so as you can see here on the right hand side i've been training my model i've tested the model it's giving my prediction but then it says provide feedback to improve this prediction it's correct if no intents in this model match the issues if you say it's correct intent should be because if not found the right one if by selecting the right one that you expected it to find it then starts to train that model and learn you know what the information is i'm just checking the q a is the nlu experience is the same with the third-party primary chatbot well that's a question i've just come through here i'll just answer that while we're here so it says is the nlu experience is the same with the third-party primary chatbot like microsoft so if you're talking about you want to use the servicenow virtual agent but you want to plug in for example microsoft luis what you would actually have is that your nlu would be managed in microsoft luis and what you're doing is is that your virtual agent just calls that integration into that nlu so the nlu wouldn't reside in our platform it would stay resigned in the actual microsoft luis platform so hopefully that's what you're getting at there all right so just moving through uh just conscious of getting through everything um so i've got the slides out of the way and this is what you'll all be pleased to see his demo let's let's get to an actual demo so what i'm going to do is i'm very cheaply going to stop sharing for a second and i'm going to reshare on my other screen here so hopefully you can all see my screen and i've been a bit funny with this one i'm opening up my workspace my service portal and i'm actually gonna say to my virtual agent you know what what is an nlu so what is that a nlu okay so you'd like to know help what is an nlu right yes i do i wanna know what the nlu is so here it says hello dale so you want to know about the nlu well here are a few details natural language understanding is a branch of ai that uses compute software to understand input in the form of sentences so this is what i've already spoke about do you want to know more yes please so it also starts taking me through i hear you're asking how does natural language understanding actually work well as i've already talked about it analyzes the data in our algorithm training models to basically understand what the customer is asking for to translate that and provide an utterance and as you can see here i've also got a quick video for you so though just a jokey thing is that i built this virtual agent conversation took me about 10 minutes to build and it was just taking text but the point of it is this whole point was all about i just typed in what is a nlu now if i had a bot that doesn't have an nou i would have to do a load of work in the keyword search and i could get multiple different conversations showing where this nlu or what is is present and so on so what i've done here i basically built a model with sorter and says and matched it to an intent and then i then it's just following that conversation so it's just a bit of fun so in the virtual agent here is the actual conversation i'm not going to not spending any time in the va at all all i want to show you here is that this was the conversation and here under the nlu is the nlu that i created and here you will see it's got i think i put about 10 utterances in there different ways i can ask for the conversation so here you can see what is natural language where can i find an nlu show me an nlu what is an nlu so this is if you think about this this is how your customers may ask for things and i'm going to give you a little cheat of how you can find out very quickly how your customers are asking for things even if you don't have an nlu yet there's a little little trick you can do so in here whilst i've got this how does it know it's connect in the properties of the virtual age i've told it here this is important it's looking at this nlu model and it's here and it's using this intent so that is why when i click on the nlu intent it's showing me the actual intent which is called what is an nlu that's why it's showing me this nlu okay now to actually look at the nlu there's a couple of ways of doing it so there is this whole conversational interfaces tab there's of san diego and you look at this it shows me uh integrate the bot with a portal edit your bot's look and feel set up greetings and so on if i go to chat settings here it takes me into the chat setting option so i can look at my branding and my chat client my channels and so on but if i go to my virtual agent here you'll see i've got topic recommendations deflection metrics natural language now if you're not on san diego you will just go into your settings and switch on nlu in san diego now you have this conversational interfaces and here you can see that my nlu is switched on if i click on the view settings it tells me that i my nlu provider is servicenow for anybody who asked about other integrations if i had the other integrations i do have ibm watson and i do have microsoft louise as different engines to show case but as you can see if i wanted to use my um ibm watson script as an nlu i can just change but you can only have a one provider so if you decide to go with luis for i t you couldn't have the virtual agent service in microsoft louise and also using our nlu for hr for example now here you can see a little slider bar ask user if topic is correct so that's when i started my conversation it then confirmed is that what i'm interested in and this is a little slider for if ask the user if the topic is switched i mean when i talked about conversation switching is this what we want to do you can also see here my supported nlu languages now because you can see here a little filter on make sure i've got nothing so i've just got language code is here and i can take those filters out i don't need i'll leave them in for now it's fine so here what you've got is i've only got english in this particular instance i only have the english model for every uh translation pack i enable it will automatically appear in here where i can enable them or disable them so in the virtual agent itself as i said this is where we're focusing on today and the one thing i want to see here see all your nlu models in one place go to the nlu workbench so i can open up the workbench here and if you just in your example in here you if you're in already here you can just type in models and it will take you but here you can see i'm in my nlu workbench so a couple of options you've got use a pre-built model import date from a csv start from blank now if we start from blank first let's fill out some details so here i'm going to put i'm going to call this test webinar just so i can delete it when i'm finished so this is the test webinar it's the nlu model's name and what is the primary language so here you can see i'm going for english but as you can see i can build it in different languages but my primary for me is english as well as that i'm creating this for what virtual agent or search this is to enhance the ai search functionality as well but that's the virtual agent and i'm going to click next so it now says view your model so i'm going to just cancel that there so here it basically tells me this model has no content so what it is doing is it's built this test webinar modeling page where it shows me my intents my entities but hold on a minute i haven't gotten here i've not got i've not done anything this model why am i seeing nine entities well if i click that what this is showing me is my platform enabled here so as you can see these are system type these ones we give you out the box so this is a system entity for software location date number i'll come back to these in a second but in my intent section here i've got the ability to create a new intent so here i'm going to call this something on the lines of ms teams [Music] ms teams issue and i'm going to click add intent so there is my intent so there you can see there is my intent this is my model so as you can see here there's my model there's my intent because i've now got one in 10. now it does tell you you need one more intent so this one intent needs more utterances so if i click back in here this is my enablement this is where i'm placing my utterances so here i can say uh i have teams issue ms teams problem unable to access teams teams is working getting error getting error in teams unable to undo too many more and able to sync themes so i've now got a few but you'll notice there's a couple of lines under these when it says teams and what's that doing so if i click on that it's basically well what do you mean is it a software is it software communication tool or do i want to provide my own personal synonym your company's synonym so what you've got here is i'm going to say this is a software communication tool confirm so you see it's done it for every single one of these so again one of them could be the way i've wrote it why is it doing it for all of them well you can see it's just refreshed here i could say it could be something else because you could have ms teams or you could have i'm having a teams problem that could be a hr issue so you could say provide a synonym and say hr but for this i'm going to say for all these it's a teams issue okay so at this point can i train the model so it says it's never been trained i click train so it will give you an error too few unique intent found so what that's telling me is before i can publish this small model it's telling me i haven't got enough intents so i need some more in here now i can for the speed of time for today i'm going to click import intents so this is where i can import now whenever you get itsm or csm or hr we give you what's called out of the box va conversations we give you out of the box nlu models but these are all in a read-only state you can do nothing with them so but what i want to do is i can say well actually the itsm one has got some in there that i do want but i want to add more conversations to them so here i select my itsm conversations and i want to take open it ticket i want to get guess wi-fi access i want emails set up and create problem so i'm going to import them so by importing now what i can now do is bring those in to this models so they're now in there but as you can see my team's issue says it still needs utterances so do i have enough so i'm going to say teams broke teams won't start so i'll do that okay so i'm going to try and train the model again now so train so this can take a little time it's only a small model so it shouldn't take too long so what's actually happening now is it's sending this to the training server with the utterances and so on so at the moment all it's doing it's learning those utterances against that particular intent so i've now it's now trained the model so i can say i want to try the model so if i say um uh let's say i say laptop issue so i press go now what it's telling me here is you've got this create problem now where's that coming from so remember in my model itself in my model which is here i've got that problem so it's looking at everything that's in this model so if i try the model again so if i say a laptop issue telling me that create problem okay with a 61 so in other words what it's saying to me is based off what you've said with a 61 certainty i think the intent is great problem now there is one thing that always gets overlooked here when you actually go back to your test model and just move these out of the way when you go into the phases you actually have these three little dots and at the beginning of when you first created it it doesn't i didn't talk it didn't show it but if i go into the actual settings of here there's what i named it there's the language that's what it's created for but it didn't show this the confidence threshold so the confidence threshold here is basically talking about how sure do you want to be so if it's 60 when i say try model and i say laptop issue it's 61 so what does that mean well it means that i will show this to the customer because i'm passing the confidence threshold so here i can say show other predictions so the other predictions guess wi-fi access open it ticket well here's the problem because theoretically i really want that one right up there so in other words what i'm seeing here is this one is presented as a viable solution to present this one isn't so in my intent i would go into open it ticket oops cancel that open it tickets right leave in here and i would say okay um laptop issues on machine noisy say issue on laptop speaker say something like laptop not booting so that so i always remember this i actually if you could see my office now i have a sticker under my monitor that says don't forget to publish because when you you have to train the model every time you're doing anything you're adding something to it because you could sit here working away and what you'll realize is you go why is it not working why am i still not seeing that conversation getting a more higher utterance confidence level if you've not trained it once you've trained it then you can try the model then it would encapsulate that so if i say try model now and i say laptop issues so again if i okay so i'm getting that model okay so it's not giving me anything else so if i go back to my main model say try model here so it's still it's so open it tickets you can see now is shot right up so in other words the top prediction is open it ticket at 88 that problem one has gone away because it knows there's a higher percentage one in the model that's better okay so once you've done all of this you then say okay well now i've got my model very basic at the moment publish the model i can click view face and i could say here when you publish your model you make it available in the use for search or virtual agent review your model's projected performance result to see how it's predicting intent then optimize the model to reduce the percentage of incorrect prediction so you have to have this button here where you can click run optimization and you i'm not going to do now because it takes a while but it'll actually look at your model and it will basically give you uh information of how viable or where you need to improve on your utterances or where it may be a clash but for now i'm just going to click publish the model okay so it now says your model has now been published so i click got it so it's now telling me you've successfully published the model where is that then seen so this is called test webinar i'm not going to change anything here but if i just go into my virtual agent as an example so if i go into virtual agent you just go up here so if i go to my designer and i'll go to my nlu conversations right now when i go to my properties here you'll see if i load down here you'll see i've got there's my test webinar one and now it'll say okay well what intent you want so there are my intents now okay so that's how you basically build a very simplistic model okay so what happens then and this is where it starts getting a little bit more complicated so back to the model itself you see you remember these intents so i'm going to go back into my intent here and i'm going to go into open it ticket and this is very simple for me here you'll see that in the openit ticket there is an associated entity what is that this is urgency okay so if i come down here you see that this word is highlighted but i'm gonna keep this a bit funny i'm going to say my laptop is now on fire now imagine the conversational flow where if you're opening a ticket and it basically says okay dale what's the urgency of your ticket is it high medium or low like i said before my machine is on fire so it's not a low priority because if it's low and it's four hours i'll just have a desk full of burnt plastic by the time four hours have gone by so you might want the bot to sort of be able to answer that so what he'll do is first thing i'm going to do i've got my little underline on the laptop i'm basically going to say yep this is a computer so i'm just putting the synonym now here i've got the word fire so what i want to do is i want to go to a mapped entity that i've built and i'm going to click urgency and i'm going to select hi so what have i just done now i've basically just said yeah okay there's an urgent here so i'm just going to retrain the model very very quickly and i'm going to just show you what this now does in your models then i'm going to go into the virtual agent and show you how you map that question to an entity recognition so to skip the question so i'm going to go back i'm going to say try model i'm going to say laptop i'm going to be a bit crude on that laptop on fire now there's an i've done this very quickly so you can see so open it ticket remember before when i was come out open an issue and i put laptop issues it came up when i insert a couple of utterances it pushed it up to 88 now i've done pretty much the same but now instead of putting laptop issue i put laptop on fire so now it's basically telling me the intent is actually 90 percent and underneath it is basically saying because i know there is an entity for hardware okay and it's a laptop as the value and then it's got there is an entity urgency against high value so what that's basically showing me now is that that is the top prediction now what he's telling me below is remember why is it showing me this twice because i've not published the model i've published it previously and it's showing me my published model is confidence is 60 threshold and my open it ticket is 83 and i've got hardware value okay in this model where alls i did was typed in my laptop is on fire and map the fire to an entity recognition of urgency it's now basically telling me hey you know if you publish this model my confidence is going to grow and i've actually got some mappings against entity values brilliant so what i'm going to do now is very quickly i'm going to say you know what i'm going to publish that model so view phase um publish model model has been trained it's been published so now that is now published now that entity recognition where does that reside so what i'm going to do is very quickly i'm going to leave here i'm going to go to open it ticket which is this one you'll notice it's a read-only conversation remember what i said about read-only ones now i want to play in here so i think what i want to do is i actually let's say for example this is a bit of a va thing you want this va conversation but you want to edit it well very simple i'm just going to click duplicate i'm just going to call this it open ticket i'm going to call this dc for me click save now it will create a duplicate of that conversation but you'll now see that the flow is editable now what you'll see down here this is the flow of conversation now i'm not going to get into the design of the conversation but in essence what's happening here is what's the issue okay add comments and so on but it gets to a point in the conversation where it says here how urgent is it okay here now this is where you start to think about entity recognition if i don't want to ask the customer that question based off utterances they place in their mod in their conversation if i say select an entity if i scroll down i should use any look find urgency and i'm going to say enable nlu to switch no it's not that one so this one here enables switching topics as predictive manually based on the russians no this one here so i've enabled it so in other words just by selecting that word urgency if a customer comes along and says something like uh my monitor the smoke coming out of it and i think it may be on fire if it logs this particular conversation and it gets to that question point and they've said the word fire in the utterance i don't need to ask them how urgent it is because i've got a mapping in my nlu that basically says if they say the word fire i'm gonna skip that so hopefully that makes sense okay so what we've covered this morning of this afternoon or this evening where you are in the country in the world is we've talked about an nlu what the nlu is the workbench the studio and so on we've talked about what an intent is what an entity is okay so we've talked about all of them we've also this gone through the manage the model test your model and publish the model so hopefully you've got a very high level viewpoint of what the nlu does how i build one how i marry it there are other content and webinars out there where we talk about fine-tuning an nlu working in nlu in a multi-language environment and so on but the one thing i just want to touch on before i hang back where i open up the floor for questions is i said right at the beginning there's a way of finding out how your customers ask for things well there's a little plug-in you can switch on and somebody's married if you've heard of it it's called intent discovery now intent discovery is a really useful little tool there's two ways you can do it and i've actually picked one of my instances i've not done it switched on so with the power of here i'm going to log into my other instance so very very quickly so there's two ways that we can do this we can do this by intent discovery or we can also use what's called predictive intelligence clustering now i'm going to touch on this very very quickly because if you want to know how your customers are asking for things that helps you build your nlu model okay so in intent discovery just waiting for i'm just waiting for my page load in intent discovery it's a way of you very very quickly pointing at your historical records and it's looking at for example short description and it shows you against our taxonomy where they fit so if i show you in this example yeah so once i look at this it shows you how it's done so if we look at uh this was so what this is doing just a high level it's basically showing me when it loads how many virtual agent potentials i have but what's important for me is this is our taxonomy these are the intents that we use in our nlu for intent discovery and what it's showing me here is there is under the email troubleshooting i have 5233 calls which makes up 5.8 of my entire model records when i look down here you can see how they were asked for what my average resolution time was what the assignment groups were but here it shows me the top description the short description that based it on was in the show description 12 of them were coming in saying outlook email issue i've got 2.5 was saying this or troubleshooting to get oregon mail server i can see more and it starts to show me how they're asking things so i can actually say add to model so i could have say select my model and i could put it into a model and get my utterances that way so that's one way of doing it finding out how your customers are asking for things the other way is a bit long-winded but it's called the clustering framework part of predictive intelligence where i can basically run a cluster and in there find how customers how i my customers are logging calls how they're coming in via email via the system i can see them here that against for example this one i can see that this this isn't short description that he was basically saying this so i can find ways of doing it so that's what i want to do and the other thing i just want to call out here depending on what release you on we have actually just issued a little plug-in update and in the plug-in update you can also do things where if i just go to intent analytics not that one sorry let me just call it up advanced is it advanced sorry so under the nlu workbench there's a plug-in and i've actually saved it already on here it's one of these this one in the store there is this new nlu workbench advanced features when you install this particular plugin it actually gives you batch testing conflict review so you can start to basically see here and again this is an example i've got here under my itsm virtual agent model when i click on this it's telling me i've got uh i've got moderate i've got 16 moderate issues so here we found overlapping utterances in these two intents so critical can i remove user from distribution list under this one but it also exists under this one so it's basically saying you know what you've got too many models here with similar utterances if they're all published it's going to start getting confusing you've also got this option of batch testing which again there is information on that and then you've also got this expert feedback loop and i'm going to say install the plugin check out the documentation here to resolve data availability there is a more document to process no problem so if we're in the let me go back if we're in the nlu bring the models so you'll see uh is it this one i've got it in okay there you go so you see here i've got and this is an example itsm for virtual agent here i've got english french and german okay so as you can see i've got english if i click on that one and view phase there are my utterance there my intents and if i go into this one there are my english utterances if i go back home and i look at itself here converges french you'll see here i've got again i've got out of office you'll see i've got french here so if you've got multi-language let me just go to this instance if i go to virtual agent and go to a conversation i should have i should have covered this i'll do it now so if i go to teams it's one of my conversations i've built in another instance and this is probably going to be in the wrong scope yep let's go to the right scope yep so in here now when you look at my conversation here when it loads there's my nlu okay and here is my languages and under translation the translations is the translation for the fields in the designer the nlu mapping is different so here i can say i've got english as my english one if i then say i want to do it for french and then say here is my i want to use itsm french and which one i'm going to use let's say let's say it's this one for now okay so that's my nlu model nine languages so this one here so there's the nine languages defaults english so there is the french one so in essence you would have to create multiple intents against the languages so if you wanted to create an english model write an english one create a french one well then what you do is against each individual conversation you can see here that you just basically say okay and now i want to add in the german one now and you could add in another one as well so you can see there's doing it for german so you do have to build the models multiple okay okay thank you welcome doesn't help so we've done that one done that one uh hello how to train oh yeah we've just done that one uh is there any approval process for intent describing to add internal you nope if the admin has got permissions to an intent discovery and they add it they have to manually add it to the nlu then you know it's done but apart from that there's no uh permissions to see so even sorry i just had one more question uh so how do we generally implement this like do we train uh test and publish in each instance are how the implementation looks implementation plan for mlu so again what you're going to look at is with your nlu you've got to look at how many how many what virtual agent conversations were launching how many languages we're going to have to provide it for so that means that if i've got 10 va conversations and i've got three business languages then i would have to create three nlu intent three models with the different languages and map them against that one conversation that dictates how you would deploy so building your conversations building you your models your intents and your utterances in the relevant languages and then mapping them to that potential conversation so when we are migrating all this will be captured in the update set instead i'm gonna i'm gonna have to i'll i'll come back to you on that one because there's a couple of intricacies that i have to be careful of so i'll come back to you on that one yeah thank you thanks all right so if we've got some questions still going we'll collect them and i will answer them back and i'll get we'll get them back to you um because we are running out of time so if if i've not answered your question we will get to it i sincerely hope you enjoyed the session um and hopefully i'll see you soon i'm just going to share my screen very very quickly as the marketing team will shout at me if i don't so i'm going to just share and just basically say thank you for joining us uh we did the questions remember that we do have uh other webinars planned meetups on upgrades itsm hr item and more with that i say thank you i hope you enjoyed today's session and i hope to see you again soon take care and thank you from servicenow