good morning good afternoon or good evening depending on when you're calling from hello everyone name is Christopher Elliot as always um and welcome to session two of the S so modern change lifecycle launch and learn today we're going to be focusing on success scores and because we we really pressed for time the last session I'm gonna hurry up and just pass this off to Greg and Isaac and they're gonna take care of you today thanks Chris yeah um thanks everyone for carving time out of your busy schedules to to join these sessions and really learn about some of the new features that we're delivering in the Next Generation change product um I know everybody's busy so thank you and it's really cool to see this many people excited about change management so as Chris pointed out last time there's a safe harbor which I know all of you have seen um I'm thinking in this deck there is nothing forward looking um but we will be talking about the capabilities and kind of how to get up and running with change success score uh actually success scores in general not just change success scores so for those of you that are uh following along the Journey of the whole launch and learn series with change management um we're super excited to have you in these and the opportunity to talk to you about some of the cool capabilities we've been working on so where we are today is um session two we did modern change on May 24th and I think Chris posted the recording and today we're going to be covering change success score Change Model success and we'll bleed over just a little bit into some of the risk capabilities as a setup for the June 21st discussion all right oh sorry about that I clicked the wrong link there um so for those of you that were in the first session I'll be really brief about this but we talked about um really the Journey of change Evolution um looking back we're clear back to in the 1980s and 80s and 90s when we saw itel version one start to emerge and as a result we saw Change Control boards get stood up and they were very much um designed to really stop the hemorrhaging of change related incidents in the Enterprise and what we saw as change managers put in a kind of a precarious role where they had to really act as that final safeguard for production but as we evolved through both the iil worlds and uh change management in general we saw more of a change management practice um this was this really did well for a lot of organizations and Enterprises around the world just having that found best practice Foundation from itel um the problem with it though was very much around that itel really didn't address anything other than the operational side of the business so it left a fairly large gap there and I think we've all heard this story from change managers over the years of uh particularly reflecting back around you know if it doesn't fit in the itel process um you need to to change the way you work so that it does fit in that uh where we are today in the kind of 2020 plus time frame is we're seeing customers move to more of a change enablement model where it's really about partnering um to ensure your change requests are automated safe and delivered without DeLay So the goal is really to use datadriven automation to increase change velocity so when we start talking about success scores when we first started working on this about two years ago or so it was really exciting we were talking about you know change gamification and all the cool things we could do to really make change management more exciting but more importantly we wanted to look at how can we leverage some of that data about how well teams are doing or how well an organization is doing with change management so is it really change gamification I think you could argue that or I would argue that it's change enablement chess and the reason I say that is we're very much if you remember we're trying to get to this place and help get customers in this place where it really is a datadriven approach to change that drives High Velocity change into the organization while minimizing those change related inci incidents while also maintaining governance in the organization so um I think of this very much like a change enablement chess game so let's talk a little bit about success scores and what we've done here the first one is change success score um we delivered this back in Paris I believe and it really is think of it like a FICO score for your change teams that gives you um a representation if you will a numerical expression of that group's change success history in terms of where they're um targeting changes so it can be used to evaluate the likelihood of future success and help determine the level of change rigor required and it also brings a little bit of fun into the change management organization but what's really cool about um change success score is we very intentionally built this in a way that customers could extend it and modify it and make it fit for purpose in their organization in fact if you look across the portfolio change capabilities that we've delivered off over the last couple years we very much tried to bring it down to the change manager the the administration pieces of this but also really Drive um the ability to be extensible in these things and tailor your change um process or life cycles to the needs of the organization so with this you can actually change these bands of what's high medium and low um you can modify these success metric multipliers um they're using PA formula indicators on the back end and you can add your own indicators in here as well and when Isaac does the demo in just a few minutes um he's going to show you that by clicking on this view change history details you get a basically a view into a leaderboard well first off additional analytics around that group but you also get kind of this leaderboard concept of being able to see how well the teams are performing across your entire um Enterprise ecosystem uh what we found after that was there was an opportunity for us to add Change Model success and what this is is it really looks at you know how good or bad an organization is at delivering these change types or models um so we've always had this with standard changes where we M we um rendered the success rate of those standard changes but now we're rendering that success rate on all the cards so that you can get a good view of exactly how you're doing with those types and models and I'm purposely saying types and models here because um even though we're calling it change model success we support both types and models in this scenario so you can get success rates on types and models depending on how you're rendering the cards so what we did is we took the combination of these two and rolled them up into a brand new module called success probability um and we're not going to dive too much into that today because we're going to be covering in the next session when we talk about next Generation risk but I do want to call out the distinction here just to make sure we're all on the same page about what this is and isn't so change success score is very much a representation of the team's success regardless of what change types or models they're using and then if you look on the right hand side over here with Change Model success we're really looking at how good or bad is the organization at delivering these types of changes into the target environment regardless of what team does it so when you start to bring these two elements together you get a much more succinct view of exactly what the success um probability is of a particular change that's coming into the um Target environment so let's talk a little bit about how this feeds into risk because this is is really an important element and um it's important to note that like I mentioned in the in one of the first slides is change gamification is one piece of it but we're really it's really about capturing that data and leveraging it to drive things like to act as an input for risk or to evaluate approval policies so with the Legacy risk scoring that we had in change management prior to Tokyo it was basically based on a single condition Builder we rolled both risk and impact up into that condition Builder which added a little bit of complexity for a lot of customers um trying to make the distinction between risk and impact uh we never really evaluated success probability and we would triangulate those risk scores and derive the highest risk value and then there was pretty much uh or not pretty much for a lot of organizations there was a pretty big dependency on those Legacy risk assessments and I think most of you know at this point those risk assessments are very subjective in nature and where we're trying to bring customers to with change enablement is really to get more of a datadriven approach where we can remove some of that subjectivity from um the approval process and the evaluation of risk so with modern risk scoring we took a look at you know what does the industry look at when we talk about risk not just it but just across the entire industry and it's really based on impact and probability so what we wanted to do with this particular capability was to um drive those same tenants back into what we were building from a risk perspective So within this success probability module you can evaluate risk at um individual team levels or group levels and you can use different parameters to profile that risk so for example you can use team success probability which is your change success score or you can use Change Model success which is basically how good or bad are we as an organization at delivering these changes into the target environment or you can use a combination of the two to basically come up with a value and what we did here is we basically ingest the current. impact value directly from risk conditions and then we pull in this success probability data um based on how the customer has configured it and I'm I'm kind of trying to emphasize that message because as many of you know we're driving towards this model where customers have the ability to do Federated change mean I can evaluate risk differently for my devops team versus my SRE team versus my uh infrastructure teams if I choose to do so and that the reality of it is those changes can be slightly different and you may want to evaluate risk differently but the important part of this is really this um the way we're doing this if you think about the way incident has behaved almost from the beginning they they never set the priority of an incident they always evaluate the urgency and impact derive risk or I mean uh priority in change we're doing something we've adopted a very similar approach where we ingest probability we injust impact and then we derive the risk value based on that so we're creating an output for that and then we've got um another capability that we're going to be talking about in the next few weeks which is risk intelligence so this is an ml capability that's available for All itsm Pro customers and um it's a incredibly powerful capability if if you have not looked at this I would highly encourage you to take a peek at it because what we saw in our data in the lab environments was that we had enormous success or Precision with this particular um capability meaning that we were seeing results in the 90 plus percent range in terms of accurately assessing risk using ML and then a lot of you customers have already adopted our using risk assessment I think last time we took a look at the numbers there was about 5,000 customers that we're still using risk assessment we are making a recommendation that customers start to kind of think about more of a datadriven model towards evaluating risk and start to move away from kind of using risk assessment as that primary mechanism to identify the risk of a change um what we'd like to do with this in the future is basically be able to hang actions off of it so think about like you know based on how I answer these questions I'm going to make a firewall change um okay you need a Security review but at the end of the day what we're doing with this is we basically bring it into risk scoring and then we always derive the highest risk value so again I wanted to kind of paint the picture of these success scores and how we're ingesting them so that you can use them in risk and there's one other very important use case that I want to call out and Isaac is going to show it to you in the next few minutes but a lot of customers also use um the Su the success scores to work with approval policies as well so if I have a score above 700 and I'm very good at it I may want to Auto approve lowrisk changes if I've got the exact same change and that team is not so good at delivering changes into production I may want to tag approvals onto it so it gives you a lot of flexibility to use the underlying data structures and past history of changes to come up with an outcome that makes sense for the organization so with that I I'm going to turn it over to you so you can do a live demo all right thank you Greg all right so success scores what I want to show first is actually another way that we can consume the data of these success scores and that's going to be underneath here under the change approval policies so you'll see change approval policies uh referenced quite a bit throughout these sessions this is um make sure that clicked it right go so change approval policies are going to reference a flow and inside the change approval policies is where you set the parameters of what you want to happen to that flow and the change ticket so opening up our normal change approval policy I can see a couple of uh policy inputs first reference these policy inputs are critical in deciding the direct that the change ticket will go so things like were there any recent outages and how long ago were those outages in place are there any incidents open on the CIS that are connected things along those lines so these policy inputs are also were available to get them thirdparty and we can pull in information you'll see in a future session how we add policy inputs from uh external systems as well to help service now make decisions these decisions are um a a collection of those policy inputs and we're going to look at these first two so this change ticket could be a low risk and considered for auto approval the next one we're going to look at is low risk but something will be a little different and it won't be Auto approved so the lowrisk auto approval has these conditions underneath it pretty straightforward so we have to have the the State at assess the change risk needs to be low no outages and then the change success score metric is what we're going to focus on for the demo has to be greater than or equal to 700 so that FICO score of the and the performance of the team plays a role in how you decide uh the way to handle these change tickets so that's for auto approval let's take a look at peer approval and there's one difference which is the change success score metric is less than or equal to 699 so that's our threshold and allowing certain teams to have Auto approvals or not so let's see that in action I'm going to go ahead and just create a new change request I'll be normal because that's the approval policy we had been looking at and I'll just set up some uh some functions here so it's going to be a low low risk so it's change in documentation and put that in there um we will change the state to assess so we do want it to be moving to that first group and then what I'm going to do here is uh pick a team and this is going to be the technical services all right so the the assignment group it's Technical Services you can see uh everything on this screen we'll go ahead and just save that bare minimum required here so we're expecting it to be at assess but look at that it's actually gone forward past sorry to interrupt but do you mind showing them the change success score model here yeah yeah that absolutely was going to do that yep so the reason we see that is the score card associated with that team so there's there's that connection that connection back to the approval policy and what decision will be made this one here is an excellent representation of performance in this in this organization so even though a score of$ 699 would still be high uh We've set our threshold here to be over 700 this certainly meets that at 806 we push that forward through past that first level of Auto approval so click on success score analytics I can now see how this team is performing in the breakout of of what's going on here so over the last 30 days as you scroll through these are the number of changes successful successful with issues but that gamification can really kick in and um not only help your company identify uh opportunities for improvement but if I go ahead and just clear that filter this is now my company score not just for that one particular group and now we've got a leaderboard here so we do see Technical Services support has the highest and they are at the top and the other team scores flow down and aggregate together for the overall company score anything else at this point Greg that you wanted me to point out no it looks good I'd just like to to point out there are a number of questions pouring in the chat uh not sure if you guys want to try to address some of those as you're running through the demo or maybe look at them afterwards so Chris uh thanks for calling that out um a few of our Engineers that were working on this originally are on the call with us today as well so they'll be responding to some of those questions um in the chat as well beautiful thank you all right so we want to um take a look at at that functionality in service operations workspace right so how does that look and feel inside the new UI of of service now so we've got multiple uh tickets that I can look at here in the assess stage we'll go ahead and uh just create a new ticket for ourselves underneath the normal category so very similar to what we did here so for why Isaac is doing this what he switched UI here for those of you that have not seen the new service operations workspace which is kind of our reimagine change experience that was released in February it's very much rolled up around jobs to be done based on the underlying State model so he's that's why the difference in the UI right here y absolutely all right so just going to get in enough information here and at this point we've got the description and an assignment under underneath a technical services support organization so I'm going to go ahead and calculate that risk go ahead and move this through we can see uh another representation of of that scorecard coming through but also showing us that the risk is low probability of success is high the impact is also low and that change success score is part of that calculation so this team not the individual but the team has an excellent uh rate of that change success so we've got a 90 90% score there so I'm going to just show well this is going to actually stay across all of the stages as we work through it here so even though this ticket has already been created I'm going to reassign this keep an eye on on this uh this risk calculation so when I reassign it um it's still going to be to Beth England but at this point here we'll switch over to a different team and that she is a part of which is the itsm Engineering Group go ahead and save that and just on the Fly you noticed we we did see the change success score we still have a high probability of success that's because this is for some documentation change but um the change success score will be put into that calculation and it stays in place all throughout the states of that change ticket all right that is that is H what I had there Greg yeah so can I just make a couple comments here Isaac so before you stop sharing your screen if you can leave it up there so one thing that's important to note is um when you're in s so it's important to run this recalculate risk action that you see over on the contextual sidebar there once you've changed the assignment group because what it's going to do is re-evaluate risk at that point so even though Isaac showed you that simply changing the assignment group will change the change success score we actually have to recalculate the risk and he just did it there and now that risk is moderate so why why am I talking about this um the reason being is we're using the exact same change and what we've done is basically just assigned it to a different group and the risk has moved from low to moderate as a result of that so you can see the power of leveraging these success scores to evaluate risk and if you haven't taken a um a look at the new service operations workspace we would love to have some customers jump on there and take a peek at it you can get it on the store um it's it's really powerful and it's it's really going to change the way customers do change management for years to come and it simplifies that experience for your users anything else from you Isaac or do you want me to take over oh yours great okay um all right so let's talk about um a little bit more in detail about what's happening here and I promised Chris at the beginning of this that I would keep the presentation part of this um somewhat succinct so that we could answer as many questions and address things that you guys need to know from us um so I'll try and go through this pretty quickly um the delay data collection we're looking at total changes successful changes unsuccessful changes and keep in mind when you're looking at these things things I think most of you will that work in change management a lot will recognize that some of these are based off the close codes and the change and the incidents caused by change so that's the data collection we're doing then we bring that information in and we do these uh there's success multipliers in the formula indicators inside PA and basically you can say things like I I think out of box we ship it with um if you have successful change with issues I think you get um maybe one point or it might even be a negative I don't remember off the top of my head exactly how we set up those multipliers but you as a customer can go in there and adjust those multipliers so that they make sense for your organization we took a uh a pretty good pass at this to try and think about what customers would need and what multipliers we would apply so I think we hit like 80% of it but for you customers that want to adjust these and make them more tailor to your organization you can certainly do that so let's talk about activation and how this works a little bit so first off for plug-in activation you can go and activate the change success score plugin um and even though that says change success score we're actually uh this second plugin gets loaded automatically this change success score Foundation but we also Incorporated Change Model success into this single plugin so by loading this first plugin you're bringing in change success score and change model success and we also added this um success probability module so if you want to use the advanced risk capabilities that we're going to be talking about in the next session you'll want to load this um plug-in as well in terms of access restrictions um I know one of the questions on the last session that we did was help us understand which of these are pro features which of these are standard um for Success scores these are very much Pro features um in part because they're using PA on the back end but the access restrictions are really around um you know based on the customer PA configuration and for the success scorecard that Isaac showed you it's really the ability to read the assignment group and then the dashboards um the roles you'll need are iil and um SN change read and how it works uh let me see if I can clear my screen a little bit here so when you load the plugin the PA jobs get added to collect change uh the success score data and by default I think we have them set up to run at 2 A.M um UTC or and you can basically adjust those as needed or configure them to make sure they align with your environment but one thing I want to call out here is that as we mentioned above uh in the previous slide we're really collecting that information based on close codes in your change request with an and changes with an actual end dat of the previous day so we're looking at the previous day data and what the success or the close can um uh code was in that change request and then were there any change related incidents associated with that particular change once that job is complete there's an event um that gets fired that triggers additional processing so this additional processing uses a PA formula to calculate the adjustment needed to each of the group's success scores based on the data collected from the previous day so again when you're looking at change success score you're looking at the previous day score or over a period of time because it's incrementing and decrementing it based on your change activity the score adjustment for each group is then applied to the latest score for that group and it's found in the metric table and a new metric is created reflecting that group's score as of yesterday and then there's one more final step that happens in the processing and that is really this automatic execution of a PA job called change success scores for today and this really collects that data from the the metric table so that it can be rendered up in the success score dashboards now the dashboard that Isaac showed you was for change success score we built something very similar for the change model success um and what's cool even though we're not going to get into that today what's cool about that is you can use the breakdowns in PA to look at different categories of changes so let's say you looked at normal changes in my organization are running about 90% um success rate on them so if I want to look into that in PA I can look into that normal change and now I can see oh here's one category of changes that are performing well below the other ones and bringing my score down then you can start to dial in and look at that and see what's going on with those potentially even create a different change model for them so really um important capability and we Tred to mimic the data collection and the render ing of those values in a very similar way across change success score and change model success to make it very easy for um change teams to be able to administer this all right so questions and possible answers for those of you that don't know me very well this is my attempt at humor so uh questions from anyone that yeah we've got a few Gregs so one question uh that's come up can the success score calculation be based off of problems with cause code as change this feature is for change management only today let's go back here see what we have all right some of these questions will'll we'll need to follow up in uh text format so things like product documentation for the success scores uh and samples of those how you can do the calculations so we'll need to follow up on that with our email response yeah and by the way for those of you that put questions last time in the chat I think Isaac and I went back after the session and and responded to all of them so I believe Chris sent that message out along with the recording on Modern change yeah so if you asked a question and it didn't get answered in the session please refer back to that we're going to try to do the same with this if we don't get to your question in this session we'll we'll follow with a response for sure I'm recording everything that happens here guys so um I won't be dispelling any personal info identifiable info but all your questions will be answered also Michael John I saw your hand was up did you want to come off mute no okay all right we do have a question m I'm GNA put the thanks I couldn't come off mute so there you go so I I POS a question in the Q&A section and I saw Isaac responded I was trying to figure out for the P1 P2 P3 uh incidents that are used to as used for input into the success score how was that tied back to the change request because what I didn't see in the demo is I didn't see a a configuration item selected so just trying to understand that correlation between the incident side and and the change side to impact the the success score yeah it's a great question um it's it's basically what we're looking at is that related list of incidents caused by change on the change form and that's where we're pulling that data from for incidents caused by change now we've had a number of customers ask us about um you know some automated way to collect those incidents caused by change and a few releases back we release the capability and Innovation Labs using ml to detect those incidents automatically um and then bring them back in a way that would um minimize the amount of intervention that a change team would have to do to identify incidents caused by change and what I mean by that is when the ml prediction came back and said these are the list of changes that or incident that may have been caused by this change we allow the the team to decide which ones are or we can set the Precision thresholds with them um that feature has not been released and we pulled it back out of innovation Labs because there was so much variation in customer data that the results were inconsistent um we want to come back at that and see if we can automate that so that we can automatically populate that related list for you but today um the the short answer is you need to be a you need to be able to capture incidents caused by change into that related list to basically render up in this change success score okay so and the reason I asked um we've configured or modified our incident form to have a field to to denote caused by change and we use that to calc you know as part of our calculation of of risk so I was just trying to figure out if the P1 P2 p3s have a field that can denote it will success score Pull It in automatically or is there another way we would have to go at it yeah I I for now um I I would suggest um you know you can either modify the way change success score works today and try and pick some of that up but we don't have an automated way to pull those incidents caused by change into into the change related list um we want to get there so I I guess my um kind of recommendation here would be like if there's something you can do in the short term to address your need I you can probably do that I do believe at some point in time we are in the near future we're going to address you know capturing some of that information automatically and playing it in the change record so my point of that whole dialogue is really you know just be cautious about um customizations that you make there because we very much want to deliver our capability in that space we're just trying to find the right combination to make sure we have the right degree of accuracy in it okay yeah moving on we uh have some questions that I want to address and then we'll get to Bernard and Satish with their hands raised so are there plans to expand the success scores to something else than a Simon group I think we already touched on that in the chat but uh that at the moment is where it is Greg do you know if there's anything in the road map for yeah yeah I I can address that so we we might um we we've at least considered the possibility of extending it and potentially even um offering a property based kind of approach to reaching deeper into the assignment groups like who's doing the change that kind of thing with the understanding that there's definitely some HR considerations in there depending on where you're at around the world or even in your organization so if we were to extend it to drill down deeper into things like indiv ual levels we would definitely make that a property driven solution so that customers could opt in or out on it we would not um kind of push that out is it going to expand Beyond um assignment groups at this point we don't have any intent to do that um if if there's a good use case I'd certainly love to hear about it and we I could talk with you about it all right um let's see go ahead Bernard we're going to go ahead and ask you to unmute and go ahead with your question hi good morning so um Greg that's one of the things you were talking about with the potential additional uses for use case um our company specifically does have a use case around this and I think a few others from what I'm seeing the chat have the same or something similar um our assignment groups are really more like implementation groups so we have some separation and areas where developers of our configuration items do not push those changes to production we really have the assignment group who are the real implementers of that um and with that in mind the change success score Unfortunately today does not really provide enough value for us because the implementers are only asked and doing what they're asked to do um so baiting rating their success based on what they're being provided doesn't give us a real clean view of what is the true success issues around um these implementations because those implementation groups could be working on multiple CIS in our environment um it's not just dedicated to one there could be like 50 or 100 different ones that they may be doing these implementations for um especially like a DBA let's say like a SQL team right like they're going and doing updates to various databases on various configuration items so for us we were hoping in some way or maybe as an enhancement coming in the future that this would be rated more around CH around those configuration items specific Al so we can actually gauge the success of those CIS based on the changes that are being done to see you know how successful was these last 30 changes that this one CI had how successful was the last you know 50 changes that this CI had um and I think that would provide for us a lot and maybe for other groups here too a lot more view of the reality of these successful changes um in their environments yeah that that that excellent use case Bernard and like I let me kind of give you my two cents on like because we have thought about some of this like we we intentionally at least for right now and I would love to have a follow up with you and chat some more about this but here's how we kind of approached it is when we were looking at some of the underlying risk capabilities using ML and you're probably thinking why are you talking about risk I'm talking about success score but I'll Circle back and tie it all out is we wanted to incorporate like stability of CIS and um activity around CIS and bring that in as an input to risk um as opposed to treating it as a success score so I could be persuaded to kind of think about it differently so that's why I want to chat with you some more on this but at least the way we've been thinking about it today is you know we want to collect some of that stability information and other details around risk um at the CI level but we wouldn't actually pull that up into the success scores um right or wrong that's kind of how we approached could we do it differently in the future for sure um it's definitely worth chatting about yep and and happy to have that conversation Greg so um if you need some details from me I'll provide it awesome you know talk about that future yeah I I'll follow up with you thank thanks that good use case though I I tend to agree I'm just not sure we're LED on where it should render but we we can talk about that understood thank you all right uh a couple questions here I think I'll I'll type that answer out so Satish I'm G to ask you to unmute and floor is yours hi um good morning guys uh this is a question for Greg uh I think like when you are showing how to define the risk policy within the Tool uh is there any option to include the type of change rather than other parameters like because as for applications in the cloud there may be a minor change but it may be impacting like many other applications So based on the past successes based on the incidents based on the P1 P2 I don't think like that really gives the real risk of the change that we are implementing the real risk is basically on defining what type of change we are doing right so did you guys think about like that in implementing this uh new risk scoring polic risk cing framework uh instead of like going with the past history or the present history are we can we not like include into the type of change that we're implementing into the CIA which will Define the risk score yeah excellent question um so the next session we're going to be covering that in a bit more detail but if you're at the beginning or towards the beginning of this I talked about this concept of Federated change where you could basically apply um a risk profile differently for different teams in the in the organization or even different scenarios or use cases as part of that you can very much um you know trigger a specific set of risk evaluations based on the type of change you're doing absolutely that that's supported today as of the Tokyo release you can evaluate those um independently based on the the type of changes they are so you could say like um normal change where you know category is document ation and what whatever the case may be or maybe it's not a normal change it's some other kind of change so absolutely that flexibility is there today as of Tokyo all right Brooke we uh let's see go ahead and ask you to unmute and we're yeah thank you um my question would be I guess it's kind of a two-parter question in the beginning you mentioned how we are able to add additional indicators to this however in our instance we have seen that if you add additional indicators they don't actually pull into the change success score would it be possible for you guys to set send out like a demo of you guys creating a brand new indicator that links up to the change success score and how it plays into it um and then I guess the second part of that would be if it's if it's not specifically change related so let's say it's it's we want all incidents we don't just want those that are change related we want the teams to be impacted by all incidents would that be something like an indicator that we could set up or no it has to be change related um Posh you might have to help me out here on this one I don't I think we can pick it we I'm not sure I'm not sure I know the answer of like what that would look like if we picked up all incidents Brook like what what are you thinking like all incidents that happened over a time span I'm just trying to make sure I understand the use case Brook do I are you still there oh yeah sorry I got muted um yeah so I guess for um we wanted it to be um not just specifically change related incidents we wanted it to be all incidents um so that the teams get impacted by all incidents so it's we we took off that condition of that change related and now it no longer plays into our change success scores and our change success scores are no longer fluctuating okay let's do this Brook um can you can we re uh contact you afterwards and let's talk through this a little bit more I just want to make sure I understand that but I want to Circle back to the first part of your question um so can you add indicators we just um we just did a short write up on how to do this we don't have a video for it but I can share that out with this group um of how to add indicators to the success metrics and um we could potentially create a video around it as well showing how to do that so I I I think that's a good ask so okay but but the second one I think I need to follow up a little bit with you just to make sure I'm clear about you know how you would look at that and so that we can help with it okay thank you sure all right deepok we're ready for you thanks for that this is just a clarification for the Q&A I posted in uh in that section so my question was can we alter the fabric of the uh chain success score calculation by replacing incidents caused by change with problems caused by change and the reason I say that is because a lot of time those incidents are resolved by teams which are not the root cause of the problem or the teams which have made that change so that probably is a a strike against them when they are not even at play there um you you should be able to change the jobs to collect problems caused by change but I'm trying to think I'm not sure off the top of my head if I remember and Le or TSH you can jump in here if you know but I don't know that we have a related list for problems caused by change I know we have incidents caused by change and we have problems fixed by the change but I don't recall seeing a related list out of box for problem maybe it's not out of box maybe just our own shop but if you have the data yeah Tac if you guys have that data you can just change your indicators and pick that up yes perfect thank you yes all right Franco Franco okay you can unmute yourself if you need to all right so we've got a question here Greg do I understand correctly that there is a PA dashboard for change success Change Model success scores similar to change success scores right now do you hear me yes we can can hear you sorry sorry for that uh well thank you my question is how much time is being considered for for hisorical prach indicator uh it means maybe at the beginning some team it's doing a very bad job in implementing changes but if they improve their performance in the last three months or something like that I I want to know if there's some specifical time that we are considering to to get that metric in order to maybe at the last three months they improve the change success scoreboard because at the beginning they had a very bad historical maybe something that it's going to affect that kind of evaluation so my question here is how much time is being considered for for this toal for for these indicators yeah it's a good question so here's how they work um every team so think of it very much like a FICO score every team starts off with a score of 500 for every successful change they do it the score increments for every unsuccessful change they do it decrements um so it's almost like a running score we have a historical job out there that gets loaded as part of the plug-in that you can execute that tells it how far you want to look back when you in um basically activate change success score so you can say I think the Autobox configuration I think is 30 days if I remember correctly um so what happens is you basically your score goes up and down and adjust so if your teams start getting better you'll see a trajectory on the model that shows that you're you're getting better and improving your success scores the thing we did not add um and we talked about it but we didn't add it was the ability to basically reset um and zero out the scores again um if if there's a number of use cases we could certainly entertain that that idea of doing that today it's not there but so essentially what you would see is the indic or the um you see the trajectory moving in a different direction the team is improving in your example um and their score would improve over time but there wouldn't be like a zero it out and start over kind of thing thank you thank you all right Angela I think we've got time for maybe one or two more hi I um wanted to ask if there were other elements taken into consideration around like compliance U where teams do things that maybe they implemented without approval or we have undocumented change scenarios where uh we have some compliance items we have a compliance tab will we be able to pull things like that in to the uh score as well yes you should be able to just create another indicator that's with those success metrics Angela you can add the success metrics in there so you know if you wanted to add something like that and adjust the score accordingly you could do that um one other thing I just want to comment on that or two things please um One is we do have an Autobox unauthorized change capability um that um it works it works well it's not ideal but it works really well um and that would flag changes that are happening that do not have a change request associated with them as long as the CIS are part of an application service that's an important distinction there to to be aware of the second thing when it comes to compliance and I just want to make sure you're you're aware of this um today the current plan and I I mentioned early on with the Safe Harbor that we weren't going to be talking about any forward-looking things so this would be the exception of that so this would fall under that Safe Harbor disclosure but we are looking at um basically embedding compliance criteria inside the change models so that change teams have better um control over what's happening in the life cycle but more importantly that the teams when they fill lot of change request your knowledge workers or consumers of change management they will get um a UI back inside the service operations workspace that says here are the five things that you need to do before you can advance this change to the next state so we're going to drive compliance back into the models to make it much easier um to administer and also for your knowledge workers to navigate the change process did did I answer your question Angela all right uh we have a question here so does the starting score for those teams with a 500 that have yet to create a change get calculated into the overall organizations change success score I don't believe so I think we glude um teams that have not done any changes it may not be as of the Paris release but I think a subsequent release we added that on there um any any of the Lee or Tosh do do you guys know oh it looks like Lee raised his hand oh yeah you can unmute yourself Lee you're good yeah sorry um sorry I was answering the question in the chat window could you just um repeat the question one more time please uh do uh teams with a starting score of 500 that have yet to open a change request are they are they calculated into the overall company score oh yes I think it should be it would be yeah all right all the teams that had change request in the past will be calculated um for the change successful and then in the dashboard the uh each team Su SC contributing to the average school for the okay so do they have to have an open change ticket or rather they're are going to be teams especially new ones that have no change tickets created so if that's the case do they not get included in the calculation if they've never if that team has never created a change um if a team never created a change request in the past but if they um started having a change of request for example starting from today then the score will be calculated for the team perfect that's what we needed all right all right so we have um we've been going through chat but we're we're going to have a followup where we're actually going to make sure everybody's questions do get answered that did post a chat we thank you for that thank you very much for the feedback for the questions and we will absolutely follow up much like the same with an Excel list with the questions in there and our answers absolutely thank you everyone for joining um I'm going to post a link to a survey would love for you to fill it out if you have any thoughts after this call let us know um we do look at everything um and we really want to capture your voice um so as always if you have any questions feel free to email me again thank you for joining this is a wonderful call thank you