so as always before we get started let me share a few resources that we had available for you we have an AI and intelligence Community Forum that we post resources on and you can also use to ask questions so myself and my team is monitoring the the questions on that so it's the best way for you to get questions uh answered there and to access that there's a quick link as n dot Works slash AI if you like the content that you're seeing today you might want to join And subscribe to our now Community YouTube channel you can find that on YouTube at servicenow Community by all the videos there you'll find our academies but also academies from the other groups a lot of great information there as we mentioned previously we recently launched a get started with AI solution course on now learning so if you're interested in getting an overview of all our AI Solutions and how to get started this is a good place to start as well so that's on now learning if your customers and you are interested in partnering with us and sharing some of your data so we can help you build better AI products we have a global AI Innovation program that's ongoing right now if you want to if you're interested just use that email to get all the details of how you can partner with us to shape our future AI products today we are going to cover content that is available and that can already be accessed in your instance but if you have questions or if you were if we were to mention features that are coming in a future release please read and apply our Safe Harbor or notice just remember that when you make a purchase purchasing decision it's better to check back with your accounting just to make sure that everything you've heard is available at the time of the purchase if you are new to AI Academy here are some of the basics of those sessions this is really content that we bring for you these are fresh ideas to provide better understanding and practical guidance on our AI products this session is recorded and then we'll be posted on our YouTube channel but if you are here with us today this is a great opportunity for you to ask your questions so we kindly ask you to use the Q a panel in the zoom interface to ask your questions you can be at any time during the session and we all get to those and provide the answers as best as we can here is the agenda for today we're going to start with a quick overview for about five minutes and we spend we'll spend most of the session doing a live exercise in our instance and we'll save time at the end and along the way to answer any questions that you may have today's topic is how to save time routing cases with task intelligence my name is loik Sanchez I'm the outbound product manager on the platform AI team and with me today is Qing hey guys nice to meet you I'm James Tolkien the product manager uh in the AI group great thank you so let's get started with a quick overview of what we're doing today so your organization is probably dealing with a large volume of incoming cases the existing method to review and resolve those issues are highly manual and time consuming especially when it comes to triaging and prioritizing those cases and this holds agents back from actually working on solving those cases and then in turn delays the value that you can provide to your customers there are rules-based method that you can use but they are limited and most importantly they are expensive to maintain because they do not leverage your past data and this is where task intelligence can help task intelligence helps on that task using machine learning by helping agents spend less time on the steps of receiving and preparing the case to actually focus on solving that case and in turn your customers get their answers faster it also helps to reduce Errors By automating the repeatable steps and all of that is available without costly configuration that enables a fast time to value today we look at a few capabilities of task intelligence the first one is Task intelligence admin console it's an intuitive experience to set up and deploy AI Solutions it helps you get started quickly with provided solution based templates for some of the common use cases that intuitive experience provides the steps and guidance for admins just to set up the AI models it also provides insights into the potential outcomes and helps test predictions before deployment and after deployment you can access that console again to see the metrics and assess the performance of the models in talking about the available templates today we'll go deeper into the model to Fair case fields it predicts the values the values of case Fields saving time and enabling faster resolution by providing the context to the agent without having them to go through the whole description of the issue or the email so this allows to Route the case to the right agent based on those values so that the case is resolved faster the the best in in that approach is that it works with your data even if you have custom Choice values because it's trained on your data it provides the results that fits your own needs so that was it for the overview now we're gonna go to our instance and do some live configuration you can follow along before we go down let me just set the stage for that we are a streaming service company that gets a lot of customer cases those cases are created by our customers via a form on the portal or via an email what we're trying to achieve is to get that case in the hands of an agent that is qualified to resolve the issue as fast as possible now the problem is that today after the case is created there is a lot of manual work to try to find which category and the priority of that case and this step is needed before we can apply the logic that routes the incident to the right agent support agent queue in other words that start from going from unstructured data the description of the issue to structured data a category for example that step is very manual and this is what delays the pro the process and today we'll use task intelligence to automate that process we'll use an AI model to predict the category of our case so that the case can be routed to the agent in a matter of seconds rather than minutes so let's go into our instant all right let's do it again so we are looking at the list of our streaming service cases from that list we can notice that we are almost 13 000 of them and they are there are different categories so let's dig deeper into that and group them by category and that helps us understand the distribution of the cases between the categories and we can see that the data is fairly balanced I have good clusters of cases within each category except for the last category here the feedback category and because I've saw the low count of cases in that category I can already say that this category is probably going to be skipped yeah I'll touch a little bit on uh why that category would be skipped so machine learning works by learning patterns from historical data and if you have a very small number of cases that apply to a certain category like what loic is showing here the model is just not going to be able to learn the pattern and it's going to skip the prediction the same would be true if you were to add new Fields or add new values to existing fields for example if you were to add a new product to your inventory it will take time for the model to learn about what kind of cases belong to that product in that situation if you're adding a new product you would want to start with a rules-based approach for that new product and after some time it could be weeks or months depending on your case volume you can retrain the model and it will learn now that it has access to that data and you can then remove those rules great clarification here so now that we know that we can actually start setting up our AI model and so to do that we'll use task intelligence so we navigate to task intelligence for customer service and we go to the setup menu and that's the admin console we were mentioning earlier I have metrics about my my my live models and I can set up different models based on my use case here I'm I'm looking at prediction model so I'm going to set up a new model for that and I can just follow the steps to to do that configuration the first step is to define whether we are pre making the predictions when a new email arrives and that's based on my historical email data or if I want to predict the case on the case record itself now he if you are already bringing your emails into cases you have you could choose either option but what you have to consider is whether all the data in your email is being carried to the case so for example if you think that the subject line and the recipients is not it could be to the case and that's a variable information to make predictions then you'll want to go with email in our case we are working with the case also notice that I can include the attachments if that content is really relevant to make predictions here that's not our case here so let's save and go to the next step the next step is to give our model a name and to pick the right table so my table is the streaming service case which is an extension of my customer service case so that's the one I pick and then I can choose my output field as I mentioned I'm trying to predict the category so I'm going to select the category here and then I can set up the conditions I want to make sure that I only look at incidence cases sorry that were resolved and so for that I can use the State field foreign because this is data that I can use for my training and notice that I can also set up a time window for which those cases were created uh do you want to tell us more how to pick the right window yeah absolutely so organizations evolve over time the kind of cases you were dealing with two years ago are going to be different than the cases you're dealing with today you want to make sure that your models are learning from the most up-to-date data a useful rule of thumb is to use about the last three months worth of data to train your model but if you're an organization that deals with seasonality where you see the same cases at the same time each year then you want to make sure to include about the last 13 months worth of data so the model will have learned from last year and can apply those learnings from the same cycle occurs this year next we can talk about how we'll choose the input fields so here loex showing that we have input fields for a description and short description how do you actually choose what the right input fields are so first you want to make sure that the input fields are actually available at the time the predictions being made for example if you're predicting category you probably don't want to use assignment group as an input field because the assignment group isn't going to be available until after the category is determined so every prediction will just have a null assignment group and it won't help the model learn any useful pattern second When selecting input Fields it's useful to think about it from a human perspective put yourself into the shoes of an agent if you were trying to predict category which Fields would you look at when deciding which category is relevant short description description those seem like they would have useful information on the other hand priority is probably not that useful in identifying category so I wouldn't choose priority as an input field to recap when you choose your input Fields consider what data is available at the time of prediction and think about it from a human perspective thank you for for that clarification so now that we applied our conditions and we set up uh we Define our input fields we are ready to review the number of records and as a baseline we need 10 000 records so the model is enough data to learn patterns and in that case we are right on target so we are ready to launch the training now the training step is a step that takes some time take a few minutes and just to save that time in the demo today I actually run that previously so I'm going to switch to the to the other model that has already been trained just I want to point point out here that because I already have a model that is trained on the category field this is where I actually got that warning here so the the the admin console here is letting me know that I might try to predict something that is already created by another model so it's helpful guidance so I'll just exit that model and go back and go to the one that I previously trained just to show you that it's the same I'll go through quickly the steps so I did select case here I have the same condition separating category based on shop description and description then I trained my model and I get an overview of the number of values that would have been autofilled if my model was deployed at the time that all the cases were created so that's a lot of fields that could have been saved and then I get a sample test results of each of the output field so Jin you want to give us some details about that yeah so let's first talk about the numbers that you're seeing here 77 19 four percent what do those mean so this is telling you that the category would be this predicted the same as what the agent would have selected about 77 of the time it would have been different 19 of the time now there's a Nuance here um when it's different it doesn't necessarily mean that the model is wrong there are some situations where the model might actually choose a more correct value than what the agent shows and then the four percent is when the model didn't feel confident enough to make a prediction and it just skipped over the prediction so this gives you the high level overview of approximately how accurate your model is going to be and then if you want to see specific examples of what those predictions will look like you can go into view sample results so here you can see specific examples of cases the first one we're seeing is a cancellation of subscription in that situation the agent chose that the category was subscription cancellation and the model predicted the same thing so that would go into that 77 the goal with this page is just for you to get a feel of how these predictions will actually look it's not for you to assess the overall quality of the model you should assess the quality by looking at the previous page that gives you that high level breakdown so let's go back to that last page so when customers get here they often ask us what is good enough should I be aiming for 20 50 80 percent and I think everybody wants the answer to be one number but the truth is there's no magic number good enough is determined by several variables the first is what is the cost of getting it wrong or in technical terms what is the cost of a misclassification so I'll give you one more extreme example there are some customers who have a check box on their case form that indicates whether the case is a health and safety issue now health and safety issues they'll trigger several Downstream workflows and cause various departments to swarm together to quickly figure out and resolve the issue in this situation getting that wrong incorrectly predicting that something was a health and safety issue when it wasn't and vice versa the cost of that is very high so good enough is going to have a very high Benchmark in that situation the second thing to think about is how many options are there in the field you're predicting picking the right option out of five is much easier than picking the right option out of five hundred thousand if you're predicting the priority field which only has five values the model will probably do a great job of that on the other hand if you're asking the model to pick the right product from a list of 500 000 products you can expect that it's going to be less accurate and finally you should ask how well are we doing today if your agents are categorizing cases 70 correctly then 70 is an incredible score for a machine learning model since it's learning from the behavior of your agents and if you can get to 70 accurate accuracy then you can effectively automate that task without impacting other kpis and that would be a great outcome and if you want to know how do I actually know how well I'm doing today well some of that data is easy to get from the platform for example if you want to know how accurate you are with your first assignment you can go to the audit log filter for the last six months of cases and look for the percentage of cases where the final assignment group was the same as the initial assignment group you can consider that your correct prediction rate for other fields like category it's not that easy even if the category is marked incorrectly it could be the case that no one bothers to fix it in that scenario you can't rely on the audit log and you'd need a human to review it so you would want to pull in a subject matter expert give them an example of the last let's say 500 closed cases and ask them to Mark each one of them whether it was correct or incorrect and then you can sum all that up and you can get a score of how well you're doing today and just remember your AI learns from your data so whether it's good or bad data your AI is going to be influenced by that and it's going to impact the ultimate level of accuracy great thank you so there are a few questions on licensing we can just we have a wrap-up recap at the end we'll also make sure to cover again the licensing and for which product this is available as well as maybe touch quickly on the difference between that and predictive intelligence at the end there is one question I want to address right now somebody's asking us if we want to use task intelligence to populate certain fields on our case but those fields are mandatory in the UI would that cause issue well the answer is is yes but if the case is mandatory today and the user creating that that case doesn't really have all the information that they need to fill that case it's it's probably a bad UI right it's probably an issue with the process on your side so really that's where task intelligence can help by either removing that step of having to fill that value and just automating that yeah just to add on to that um you certainly can predict mandatory Fields it will be filled in just the same as if an agent would have filled it in um so there shouldn't be any restriction to doing that okay let's continue now uh so we are done with our assessment we are ready to move to the next step the next step is to choose how we want to show those predictions so we can either show the predictions in the case itself uh and there are two options for that it's whether the value that's predicted actually is going to feel the value on that field and that's what we call autofion or I can give the agent more flexibility and just give them a recommendation of what the value that was pretty was in that case I'm going to let them take the action to to change the value if needed and the alternative to that is to have the model only run in the background so it's not going to show any information on the form itself but it could be helpful for you if you want to further assess your model once we are ready with that step the last step is to review all of your choices and get ready to deploy simply by clicking on deploy here and so my model is deployed now uh just a quick note here if you were working on a subproil environment you can access the menu here and you have the app the option to export your model so if you click on that there's a a new tab that opens and a few steps to follow to download the XML file with all the content of that training just want to mention mention that if you are working in a surprise environment just make sure that the data that's there is up to date and really reflects your production environment as Gene mentioned the data is a very important step in that process of training so always need to be good and fresh and reflective of what you want to achieve so important to mention here and so our model is deployed and it's ready to make predictions there's one last step that I want to show you before actually show you the end-to-end process now is that I actually defined a queue in advanced work assignment now if we recall the slide that I presented earlier that step of defining which group to assign that case two is a fairly easy process to do once the data is structured right once I know that the category is an app issue or an app question it's fairly easy for me to say well if that's the case then route this case to the next available agent on the streaming app service support group so that's what my advanced work assignment Q is doing here the only thing I have to add here for specifically CSM is I need to make sure that the projection is actually complete so I need to make sure that the prediction status is not in progress anymore uh so I'm I'm basically telling that role I'm waiting for the prediction to be done before making my assignment which uh in the in the workflow here makes sense so we have that in place I do have another window here with an agent that is ready to that is available ready to work on cases and I'm going to create a new case so I'll go back to my list of streaming service cases and I'm going to create a new case I see by default the category is undefined and I'm going to input my description so in that case the customer installed the app it's not working on their TV I can simply save that case here I'll just change the view here and I see that the category here was predicted so task intelligence predicted that the category should be at issue based on that short description and because of my advanced work assignment rule immediately the case was routed to my agent that is ready to work on it so here in a matter of seconds the case was created and rallied to an agent that is qualified to work on it it means that the case can be resolved fast and the customer can get value very quickly and so that's it for our exercise today we have one slide to recap and wrap up we started by looking at the data to really understand how it looked and the distribution of it and then we were able to set up a prediction model in the task intelligence admin console and console and the steps involved there were to choose between the email or the case table and then I selected case and then I selected specifically the the case table that I'm working with then I Define my conditions I Define the cases that were resolved and then I defined my time window and then I selected my output fields in that case I selected the category field and then I selected the input fields that again considering what is available at the time the case is created and the prediction is made and really what made sense for even if a human had to do it so in that case short description and description then I moved forward and and started training my solution and looking at the sample results and doing some additional benchmarking to assess the solution once I was satisfied with that I was ready to deploy so I just deploy deployed my model and then I set up a advanced clock assignment to take that structured data and Route the case to the right agent so that's really what we covered today in that exercise um just want to touch on licensing so we have questions around what license do we need today what plugins so today task intelligence is available for CSN via a CSM Pro license you'll need to install the task intelligence plugin that is that is dependent on predictive intelligence and then I'm going to go through the questions I'll just add one one more thing onto that there were questions about why their task intelligence is available for itsm that's not currently available for itsm but in a few months it will be available for customers with itsm Pro license yeah exactly did we touch on the predictive intelligence uh versus task intelligence question sure you want to take that yeah sure um so predictive intelligence uh is great if you have people on your team who have some experience of data science but a lot of our customers don't have individuals on their team with data science expertise task intelligence makes machine learning much more of a turnkey solution and really frames it in terms that people even without data science experience should be able to understand so um if if you don't have data science expertise um then we definitely recommend you use task intelligence it's easy to set up it's easy to understand if you do have people on your team who have data science expertise you still certainly can use task intelligence predictive intelligence will give you a little bit more granularity of control that people with data science familiarity will be interested in perfect and to go through some of the questions is because I'm not sure everybody actually always sees all the answers but just a question about like having fake data or how do you want to get about finding data to to play with it so again there are a few options to do that if you have access to demo Hub as a partner you can go on demo Hub and look at the the task and tell narrative there but really if you can just test it out on your sub product environment this is the the best data you're going to have and it's going to provide value that really makes sense and you don't even have to to fake it and you can even test it out in prod there should be no harm in training models and Broad and assessing it and seeing the results you don't actually have to deploy it you can just test it you could even deploy it in monitoring mode first and just see how it works without impacting the agent experience and that's something that you can do in production yeah I'm also seeing questions here about HR Service delivery and workplace Service delivery and so we don't yet have task intelligence available for HR and workplace Service delivery but we expect to in the future all right I think we are coming on time so we just wrapped up if there are any additional questions on the product feel free again to join uh on the AI intelligence Community sm.work AI you can post your questions there we are always monitoring The Forum and we'll get back to you very quickly here we'll see you in two weeks and thanks Gene thank you