welcome to this live on servicenow webinar about activating and configuring predictive intelligence so um well most organizations get the value of um of sophisticated analytics but uh many are just starting to understand what machine learning can bring to the table so in this session we want to dive into this and as well show you how servicenow predictive intelligence can make work better for everyone and you will leave this session with an understanding what you can do and how you can jumpstart your activation so uh just a safe harbor statement about some forward-looking statements that might be uh in this presentation um so uh don't well the basic thing is don't make any decision based on what you see in this presentation as this might include or might be part of those forward-looking statements some housekeeping rules um we we prefer that you uh stay on mute uh there will be a q a after this presentation uh however if you have uh questions during the presentations there is a there is a q a button available uh in which you can ask your questions we will try to answer them during the session or otherwise we'll come back after our presentation [Music] this section will be recorded by the way it will be shared on the service now community forum after the session and you will also be prompted to fill out a short survey survey after this session because we appreciate your feedback to make these webinars better some other um comments on this webinar series so today session is part of the live all service now it's a curated event series to connect you with servicenow experts and peers that can help you deploy your products and achieve value faster so we hope you join us again at another session um we will post a link in this in the chat so you can check out the other upcoming events in this webinar series so well thanks for joining us today we we are with with two of us presenting today my name is jose von popple i'm an advisory solutions consultant for now intelligence [Music] i'm based in the netherlands and now intelligence we cover among other things virtual agent predictive intelligence performance analytics so uh we always say all the cool stuff on the platform and i'll later in this webinar i will show you a demo about predictive intelligence and today i'm joined with my colleague timo hello um thanks for joining us today so my name is timo weber i'm also an advisory solution involved on the same team as yoast so we are both colleagues i am located in germany and the wonderful city brim and let's start it today who does not remember this this is the movie about the divine intelligence matrix in the year 99 obviously this is not something that we are aiming for when implementing our a capabilities with intelligence but what is predictive intelligence about and what do we want to accomplish from a business perspective how does pi relate to artificial intelligence what can you do with it how does it brings value to your business to answer this question we first want to ask you a question in the form of a short poll what do you know about pi um please take just a couple of seconds and give us your thoughts about that so number one is i'm relatively new to predictive intelligence so that's why i'm in this webinar number two is i know the basics and i want to know more and that is i know how this can be benefit my organization and i'm looking for ways how to implement this and for us i'm actually currently in the pros of implementation or have already implemented this so happy to see some answers there's still some upcoming things and most of our fifty percent of you just that you that we have a small amount of recap there are relatively new to predictive intelligence so um thanks for joining us but there were also some folks with us who know about the predictive intelligence already and we also have two of those who joined to underprice of implementation so um thanks for joining so let's start what kind of challenges does it service departments have typically so when looking at i.t service departments one of the main challenges is the scalability of these departments you invest by hiring and training agents acquiring software licenses and have operating costs and as a return you get improved cset hopefully from the users for whom you're providing support however this roi or c-set only goes so far as and flattens out you can spend as much as you want and at some point this will not make your service desk more effective they have become as effective as they can be okay but how to solve this well actual project intelligence can make your service desk more make more effective and efficient by applying machine learning frameworks to your business processes so ai and machine learning can help but where do i start this is the second challenge where and how should i apply the capabilities of predictive intelligence should i start the complex or more streamlined where can i make the most benefits for my company and yeah if we figure out where and how to implement predictive intelligence the third challenge is to find data scientists and ii specialists to implement and tune these machine learning algorithms get them to work and apply them in my business processes but do you actually but do you actually need to have a degree in data science or get predictive intelligence to work so what is the significance of ai and ml a human mind is great at solving difficult problems dealing with new situations making hard decisions with limited facts like what can i eat at lunch when going to the supermarket and my fridge is empty finding creative solutions like pasta with tomato sauce um it's all the time available right and so much more then there are the things the human mind is not so good at we are not built to spot subtle patterns in gigabyte of data i currently process thousands of transactions or search fast databases for pertinent information and now what if you could instantly recall solution to normal problems or quickly find needles of relevant information in the approval data high stack or automatically make a current assessment such as the best place to assign a specific task your employees would or could work smarter and faster make better decisions and fewer mistakes and focus on more meaningful and satisfying work instead of repetitive tasks and for your business that could be translated to higher productivity and lower costs um increased customer satisfaction and more motivated employees and less business risks that sounds pretty amazing right now let's have a look at servicenow purpose build ai so what we have first is we have the now platform in the middle so our ai nml is built in the platform on the same layer as our technology workflows customer workflows amply workflows or trader workflows it can be used across all workgroups that we have on the platform and with their three kinds of purpose-built ai we have our predictive intelligence and this webinar is about predictive intelligence where we have different kind of the frameworks we have regression we have clustering similarity and classification and you will learn about learn more about that in a couple of seconds but we have also our purpose builder ai called natural language understanding and this is really natural language understanding processing we have our national language query for reporting stuff we have on foundation nlp or we have also something called national language workbench where you can start digging and finding out tune your engine etc and we have search and automation ai search giving you recommendation giving you trends giving you more information more results and we have something called automation discovery where you where we are using parts of the predictive intelligence to show you more benefits that you have on the platform but today we are focusing more on predictive intelligence and there we have our four frameworks um and we will start with that we have our classification framework and with classification we can predict fields based on historical data think about predicting a category or an assignment group where creating an incident or when an incoming email is finding out what is it about to which group can i send it this is really about prediction then we have similarity with similarity we can show for example similar resolved incidents to an agent when handling an incident being able to access the solution of simple incidents will help solve the issue faster lower the mean time to resolve free up servers as agent to be available for new tasks and increase cset the third one is regression this can be used to predict numerical outputs like for example mean time to resolve mttr as customers and employees are more satisfied with faster resolution times organization can improve efficiency throughout the services and improve experiences for customers by correctly setting expectations when ngtr is available services can assign resources to cases and incidents that are associated with longer resolution times that are speeding up to the processes and improving customer satisfaction and we have clustering as the third one clustering can group incidents into clusters by identifying patterns and continuously grouping similar items when you have the output of clustering this can give you insights in the biggest topics within your incidents for example do i have a lot of account logs do i have a lot of sap issues outlook issues or part word resets id changes and so on when you have those insights you can start addressing those incidents start deflecting incidents by allowing for self-service via the virtual agent find automation opportunities for your integration capabilities or can i do improve my knowledge base for example so what can predictive intelligence do for you and already mentioned some things of that with our purposeable predictive intelligence platform you can focus on your business outcomes we enable process owners to build machine learning models based on their desired business outcomes with no coding skills needed at all but for sure you can use no code local or even pro code if you want it is up to you and your center of accident that you have available allow machine learning to flex similarities across issues to quickly identify and resolve critical incidents we are able to detect for example major incidents and recommend those directly to the agents but this is not only for critical incidents we can recommend actions also for all kinds of tasks we can suggest relevant information or tasks to help agents solve issues faster like here's a resolution that could help here's a knowledge-based article that helped previously so really similar things and if you talk about resolving issues faster mean term visual repair routing assigning and prioritizing at scale will help you to reduce the mean time to repair reduce the manual work and errors and start automatically classifying and predicting specific information on task level and last but not least you can use machine learning to mine your incidents requests and interaction data identify knowledge gaps and find new automation opportunities which you can cover with integration hub or virtualization or specific information on a portal or a catalog so how to start with predictive intelligence and we choose an example the classification framework and just recap classification was how to pre to predict fields on the form for example predict the category predict the assignment group based on historical incidents um and where we can start is with three phases for that is understand your desired outcomes and your data the desired outcomes is for example um lower mean time to repair and reduced reassignments so for that you could check simply your data and see a heat map and classification reassignments it's a simple heat map for category on the one side and the reassignment count on the second side on the second scale that you can see where do you have huge reassignments um and what to do in which time frame so understand your data try to find out where does predictive attendance fit the most for you and the second step is for sure install the plugins um you need basically to start with two of them predictive intelligence this is a group the first one and predictive intelligent for instance management example but for sure we have it for incident management for change management for catalog for knowledge for case a couple of things that you can check out in the store which comes out of the box with out-of-the-box models um which really help you and we have something called predictive intelligence workbench where we have use cases available that you can simply go through see the value train the model automatically with one click tune the model and use will show you later how this is done um and then really start digging in and using predictive intelligence and using the classification framework and the third one is tune your models so if you've done the first thing it could be that you need to tune your model and here's some some examples this is a solution statistic that's available out of the box it's shown you for which um assignment group you have a confidence a distribution and a coverage and you can really see for which group you have the best value you have the best um values to start with and where you have to tune something and tuning can be done for example with the bubble chart where you can see everything versus on the left top side you can should move to the right top side so everything was in the right top quad and the best values and you can use it the most but you can for sure add more use cases so not only using the workbench with no code um you can also use the flow designer for example and integrate your incoming email inbound action that you have already in the flow designer and use the action called classification and prediction which is available out of the box there it's just a screenshot and it's described very well in the docs where you can leverage integration hub and the classification framework with no coding it's just moving the actions defining the thresholds and the action that at least should be taken so these are the three things the three steps on how you can start and i will now hand over to used and he will start again with a short question to you okay thanks teemo for this explanation about predictive intelligence um so just uh another pull or quiz question question um so how much data do you need for a machine learning model to function correctly so we we gave you a few options um what do you think fits best so 100 000 from the last two years to take into account seasonal activities or only 50 000 to account for seasonal activities 30 000 with event generated records excluded or 10 000 records with a good spread of the data you want to predict so just let some coming in and in fact everybody is is choosing for option four and um well the thing here is a bit uh maybe a bit silly question but there there is no real good answer we have a we have a sort of rule of thumb stating that we um you need 30 000 records for all of our machine learning models so if you have 30 000 records you're good to go so that's a rule of thumb but in fact 10 000 records could also be uh a good way to start so you so there is no specific threshold value the point here is that if you start with building and training your machine learning models you will be presented with metrics how good your model is performing so you can actually measure how how well your model can predict certain outcomes and we also measure if this if this is actually true so we use this as a sort of segway into um the the next slide so it's a t-moisture presenting so uh we have some um model statistics available that that you get when training a model so if you would train your model with 10 000 records you could still have a perfect model with good precision good coverage good net automation and i just want to highlight these concepts with a simple example so consider this we have a we have a picture of a lot of dogs and cats and we have a machine learning program for recognizing dogs in this in pictures and in this particular picture it identified eight dogs uh containing actually 17 dogs and some cats in between and then out of those eight dogs it identified so it should have identified 17 if it was absolutely perfect but it found eight dogs out of those eight identified dogs five are actually dogs and the the other three are cats so this is giving the the metrics and the model statistics so to say so the precision of the model would be in this case um five out of eight so approximately 60 to 63 percent the coverage will be uh eight so it's it had it should have recognized 70 blocks it only had eight dogs so 47 is the coverage and the net automation is the product of both so it's the product of the precision and the coverage so it is 62 times 47 so the net automation is 29 so just highlight these concepts because they come back in in the demo and there's actually a trade-off between precision and coverage which we'll you can sort of present in in the following graph [Music] so think about precision uh if if the solution is set only to predict when it's very sure so if you have a high precision then it will make fewer predictions so you can you can tune your model this way or if you on the other hand want to have a big coverage so if the solution should always make a prediction then uh the chance that it is it's being less accurate when it makes prediction becomes higher so when tuning your model you have a trade-off between precision and coverage and of course the key here is balance so what's the optimal setting and there are some tips in tuning this model so good thing to remember is machine learning will rarely be 100 precise or accurate or it will never be completely sure and actually our goal of machine learning and predictive intelligence is to be more precise than than our human agents so we want to optimize the service desk we want to lower this mean time to resolve we want to lower the amount of errors happening with those monotone tasks and then the goal is to be more precise than human agents and what we typically say is that if you have a model with a precision higher than 70 percent and the coverage higher than 80 that the model is more accurate than a human agent would be so actually that's where you sort of aiming for so if you have a model coming back with numbers below that you should actually start excluding classes or optimizing your model with better data so if you have a coming back to those this poor question if you have 100 000 records and your model comes back with a 50 precision and the coverage of 40 then you have 100 000 records but still um the model is not good enough and you could better do uh uh using your human agents so that's the uh the key takeaway of of this sort of technical deep dive in the tuning of our model um and this also comes back in the demo so we have something prepared for you we have a few demo scenarios available so as timo mentioned for incident management we have the predictive intelligence workbench and in this predictive intelligence workbench we have some predefined use cases available i will show you tuning how you can tune those machine learning models so when you implement one of those use cases you will also be presented with tools to tune your machine learning models we will in between see how those machine learning models work so we will use again classification as an example and see how this works or how does the machine learning model work and automatically route those incidents and we also show within the predictive intelligence workbench how you can monitor those prediction results coming back to precision coverage etc net automation so we also have ways of monitoring uh the results and you can't optimize what you don't measure and well that's actually what we what we give you we will have tools to monitor the outcomes [Music] i will take over the screen share so let me just twitch am i good timo yes all right thank you so just consider this um i'm using a very simple example of creating an incident so i am creating an incident and i will do this for famous colleagues that and i will just state something like vpn is not working now when i save this actually nothing is happening so i have no machine learning in place and what we want to do is we want to sort of predict category so we want to predict the category inquiry software hardware we want to predict this fire machine learning because i don't know what it is i think it is hardware and i also want to predict the assignment group because if i need to fill this i have to choose out of in this case 90 assignment groups i have no id i think it is service desk so i will i will save this but actually those things are better done by predictive intelligence if you have a good historical database or data set so how do we do this um i'll use the predictive intelligence workbench so if we go to the workbench we break intelligence workbench use cases and we have create new from template so we have those predefined use cases available in the predictive intelligence workbench and the most common use cases for this case for incidents or for change requests are available in this workbench and you can simply let you take take you through the guided setup of the workbench so for example if i want to predict the assignment group for incoming incidents what i need to do in the workbench is i start and as well i give my use case a name give the short description i also give my model a name and a short description and basically that's it so i could now say start and the model will be will be trained on in this case the short description uh out of the 56 000 something incidents that i have in this demo instance i could go to advanced setup to [Music] adjust some filter criteria but the most optimal setting is sort of preset and if i press start it will start training of course i already did this because it takes some time to train a model so if you have trained a model and i did this for category and for assignment group if i go to my created use cases see a few you see here live on service now so our webinar for the category and here i have live on servicenow predict assignment group so if i go in here i see that i have a trained model available so this is actually the outcome of the previous step i press start and after a while i get this model back so i have step one completed i have a model with net automation precision and coverage so i see that there's a very high precision high coverage and the product of both is net automation so this is a very good model i can also test my model so if i go here i will say okay i want to test this model and basically do the same thing as i did before vpn is not working if i would run this test i will get a result back and it says okay i am 99 sure that with this short description the predicted value is network so basically uh i can i can keep on testing and if i'm satisfied with all my tests i can integrate my model so the next step is to integrate this train model so it will actually be used for the prediction so in this case i want to state i want to predict a category for incoming instance i want this model i can apply a training retraining schedule so how often do i want it to be retrained so so that new incidents are taken into account in the prediction of again newer incidents now i'll leave it at at once and i integrate this and again that's it now it's integrated in my in my business model so i now when i create a new incident the category will be predicted i can do the same thing for the assignment group so if i do this i have again a trained model with also a lower net automation because the precision is a bit lower coverage is still pretty high i can do the same thing again so i can also do here predict assignment group and i do vpn it's not working i run my tests and it predicts software as the assignment group so i can do again a lot of tests [Music] be happy or not and integrate my model so this will be it so i can say do this model again i can apply it to a retraining schedule to it and say integrate so those are really simple guided setup steps to integrate those two machine learning models into your business process so if i would now create a new incident those values will will be predicted i will show you later what i also want to show you is that you have influence on on the model performance so i can actually start within the predictive intelligence workbench i can start tuning those values so as said before we have precision and coverage and i can play around with those so for example i can state here let's look for something i have here a very low distribution um i can for example i can maybe exclude this so if i say if i set the position to 100 the coverage will for certain for certain go down probably it will go to zero so then i exclude this class actually from my model or i'll state it like this i can i could do this for all kinds of things so i can state here the precision of hr system support is 89 i can change this to a bit higher but then again the coverage will go down so see the coverage will drop in this case so and when training the these models fire depicted intelligence workbench the most optimal setting is already preset with this workbench but if you have specific use cases where you want to exclude or focus more on precision or coverage you can do so with tuning these values now going back to my incident so let me check this is integrated if i go back to creating a new incident so i will do the same thing create a new incident let's create it for another famous colleague joe employee we'll do the same thing vpn it's not working so if i save now i actually have my machine learning model kicking in and the the network will be predicted for category and software will be predicted for assignment group so i'll see those two values predicted here and actually what i what i want to show you as well is um that you that these predicted values are also monitored so um [Music] if i'm looking to my let me check if it's still open we have a we have a predictive intelligence workbench dashboard which is monitoring the performance of those models and here you see i have my two models selected so the live service now click the assignment group and live and search style category and the net automation is predicted the precision is predicted it's showing what has been predicted correctly and also what is being predicted incorrectly so all of those things are being measured and um well the main reason uh we i want to show you this is that we had some examples of customers custom creating this but you don't so there is a out of the box capability for this which is measuring and continuously keeping track of what is being predicted and what is being the eventual output after the incident is closed so let me go to just a bit behind the scenes i will show you [Music] the table in which this is done so this is the predictor results table and you see here my my incidence this um 1073 number it's here as well so let me just show those two so here you see what is being done so you see the the solution that is being executed which stable is the prediction is being run on um currently if it's being predicted correctly but that is because there is no final value yet the predicted input so this is based on the short description the short description the two values that are being uh are the two fields that are being predicted and the predicted output value and you here you see final input value so if i would eventually for example state that this is not software but application analyst and i would resolve this so quickly quick and dirty say salt sorry about that solved salt i save this and i close my incident now if i go to my predictor results i see that we have one predicted prediction correctly is true and one is false and the false is of course the one that i changed so this is a out of the box capability so each time an incident is closed this table is being updated and also of course the predictive intelligence workbench is looking at this so this will be plus one yeah there it is so you you have those insights you see how well your model is performing and if your model is underperforming you can start taking action on this right so you can maybe include better data see where where your model is mispredicting and tune your model there um let's go back to the presentation i don't know team unto you i will share right i will take over the share exactly [Music] do you see my she's a powerpoint now timo yes we can see it all right perfect so these are the things that we uh want to show you in the demo hope it was clear um in order for you to get started with this we have a lot of resources available so we have [Music] prepared some ways or we have to prepare some resources how you can get started and activating this in your own system and we basically divided then start now educate yourself and best practice so we have a lot of playbooks available available documentation about predictive intelligence we have several labs available from knowledge also courses on now learning which specifically address predictive intelligence we have some best practices so we have a community page available we included something about how you would tune your model so we have documentation about this as well and we included some success stories about some outcomes that we had that customers so again if we we we have some uh some some other sessions available uh as mentioned in the beginning of this workshop so we we have uh this is part of a series and we have a lot of other topics available in live on servers now so i think the link will be show will be shared in the chat again um so check these out and we we hope to welcome you there and actually now the time is for us to open up the floor for questions i don't know if there were any questions already asked in the chat or q a i see one that there is are is there sample data which which one can test how pi works um i'm not sure timo if this is available um within the plugins that we have so it comes with demo data so incident management comes with demo data but um i don't know if the demo data is is i think it's just the 7000 incidents um so if you are going again to plugins and to incident management for example you can load the demo data um and you can try to um just play around with those seven thousand i believe thousand seven thousand comes with the plug-ins out of the box um but how good the results are i don't know um the best way is really just clone the production to a sub-production little sandbox if you have available um and just try it out there with your with your data instead of using sample data okay another question sorry continue to i was just checking emmanuel but this was is it okay with your question did it answer your question okay okay thank you another question was that the link to the pi advanced topics from now learnings was not working when attending so i posted a new one hopefully this is working um if not um just try to search predictive intelligence advanced topics um if you're logged in and now learning so here's again the keyword that you can look for and you will find the first thing that the search will give you is the link to this course it could be that maybe there is that you need to be logged in and we checked that already um i believe it's a service now employee versus non-employee thing so for me it's available um so we can just actual make it that better okay other questions that we can answer okay so the course is available second link okay cool then we actually update our material thanks for checking dominic if there's no other question i think we can go to the last slide used well the last one is actually that we want to thank everybody uh so uh i hope you enjoyed uh enjoyed this session about predictive intelligence um as said before that there are more webinars in this life at servicenow series so please check this out and the link mentioned in the chat and well we hope to see you again in some of our other webinars thank you thank you