hello and good morning thank you all for joining us today let's get started shall we we have a great session for you today all about leveraging platform capabilities for a smoother upgrade the Safe Harbor notices here just to remind you that some of this presentation may contain forward-looking statements and to not make any purchasing decisions based on those forward-looking statements my name is shimail adasri I support upgrades here at servicenow and I'm very lucky today to be joined by Adam Stout uh director of outbound product management for the service now platform we've got a full agenda for you today we will cover the servicenow upgrade policy we will go over the platform capabilities that you can leverage in your upgrade so ATF instant stand upgrade plans Adam has got some really great demos for us today we'll wrap up with some resources and tools that you can start using today and we've uh saved some time here at the end for Q a during today's session if you have any questions please put them in the Q a box um or in the chat I love to start our webinars off with a poll question I'm wondering what release version are you on I'm curious to know if anyone's already on Utah if anyone is on a version earlier than Rome I dare you to expose yourself in the chat thank you for your feedback I'll just give it a couple more seconds here alrighty okay so it looks like we've got most of the folks here today are on Tokyo um we've got some people in San Diego um we've got one person here who's on Rome um and then we've already got some people here who are on Utah so that's great so let's discuss the servicenow upgrade policy there are two times a year there are family releases depending on your support and entitlement you are due for an upgrade at least once a year so if you look at the box here below for our customers that are on the San Diego release with an N minus two support entitlement you are due for an upgrade in September to the Utah or Vancouver release for our customers who are on the Tokyo release which um from our poll it looks like most of you are on the Tokyo release if you have n minus one entitlement you are due for an upgrade in September as well if you have any questions on you know what support entitlement you're on um you can find this information right in your now support portal or by reaching out to your ex got another poll question here um are you planning to upgrade to the Utah release if you I know we've already got some folks who've already upgraded thank you for your participation [Music] all righty well no surprise most of you are planning to upgrade which is probably why you joined our webinar today so I think a lot of the information that we'll be sharing will be really valuable to you so here at servicenow we encourage our customers to stay current with each of the release not only do you get the latest in platform and security enhancement you also get to access new products and features from the latest release Additionally you keep your access to 24 7 365 support which includes support with specific upgrade patches and hot fixes for more information regarding the support entitlement um please visit the impact Resource Center which should be linked in the chat for you okay I've got a third poll question alrighty when are you planning to perform your next upgrade in the next month in the next two months next two to four months where it's six months in the next six months but maybe we've got some early birds here or maybe you just don't know yet this is really good feedback for us thank you all wonderful okay it looks like most of you are planning to upgrade in the next month or in the next two months that's great okay well that is all for me for now um so I'll pass it on to Adam to walk us through some of the platform capabilities you can use for a smoother upgrade great thanks my little let me uh take over sharing yeah we can hear you okay okay so let's let's talk about some of the applications that we have access to that are going to help our upgrades um I should put a little bit more filler in here but I do have one more pull uh to go through about ATF so we are going to talk about the automated test framework um and as I start talking about it if we can launch this poll um about what are you doing now for testing so we're going to talk about automated testing but I I am curious today if you were using uh what type of testing you're doing today is it just manual are you using ATF today or using some automated testing that's that's not ATF uh there's there's lots of other options out there or if it's your first upgrade and we haven't gotten through it yet so we're seeing lots of manual testing lots of great ATF testing and some other testing as well um think I can go on to another slide while we still have this open we'll leave it open for another moment um we're going to talk today about servicenow's automated test framework this is what it looks like to come through every instance has this part of the platform no additional charge to use the use ATF um to come through and what this allows us to do is to get rid of manual testing or to to augment manual testing it'll probably still be some manual testing that we're doing but what we're able to do is take our test scripts what testing we do and document those as steps in an ATF test and run that over and over again I'll go ahead and end the poll so right now we're seeing um lots of people doing manual testing um some ATF use and then and that's basically the majority is either manual testing or using ATF uh we have something in here for for all of you to come through with ATF again we can take those test steps that I have to ask users to do and I'm going to record them as part of a it has an automated test so I can run them over and over again and what that allows us to do is to get rid of to shrink the time we have to test from an upgrade when we upgrade rather than upgrading one of my sub prods sending out an email coordinating all the tests he needs to do I can just run the automated test framework and more importantly rather than just testing when we're doing the upgrade I can do this every week I can do this with every change I'm deploying we can we can virtually no cost run these tests over and over again and find those problems earlier one of the things that I like to look at for upgrades is that I don't upgrade once a year or twice a year or four times a year I upgrade every day every time I'm making a change I'm making my upgrade easier or harder and using ATF is to make it easier as we go through so with ATF there's there's three main use cases when we're using ATF or what we're looking for so one is functional uh professional testing of business logic so this is as I'm developing new enhancements I'm changing some configuration I'm I'm uh updating a flow I'm adding a new process I'm able to run my ATF at that point in time to validate that I get what I want to that it's doing what I want it to do I get the results that I'm looking for the next we have is browser compatibility testing and browser compatibility testing this is important for some of our orgs that we support Safari and Edge but not Chrome or support Chrome and Edge but nothing else or specifically where we see lots of uh organizations that have a very specific set of plugins that are extensions in those browsers servicenow can test for what we know exists but we can't test for what we don't know exists we can't test for perhaps some of your customized uh extensions for those browsers with browser compatibility testing and ATF you can run these tests these automated tests in your browser so I'm able to again document everything I want to have tested I'm going to take my manual test and I'm going to document those as test steps create tests and test Suites and then I can run them in my browser I will bring up my Safari I'll bring up my Edge my Chrome and run all of those tests and then I can I can bring up Safari and chrome at the same time run those in uh run those as well and I can make sure that everything works the way I want it to work in the browser that I need to do so really important to to allow you to be able to do that and then the third use case which we'll we'll talk a lot about today about regression testing and regression testing allow us to understand is there a change in Behavior between before and after an upgrade so the functional testing I'm testing as I'm as I'm developing I'm testing new Behavior do I get the outcome I want cross browser compatibility testing allows me to test my browsers and to make sure that they're not introducing any issues kind of out of the norm and then we can handle those uh whether we're updating an extension or we're or a browser version or we're updating the client script that needs to run and then regression testing is just flat out let's let me go this is how it works before my upgrade I upgrade this how it works after and that it's not always bad when we say failures and we'll see in ATF what you call it pass and fail but it's allowing me to identify changes Behavior some are good some are bad when we upgrade we are changing behaviors um sometimes we're fixing an issue um we're adding new New Logic that will result in a change of behavior sometimes it's something unexpected in any case those regression tests are going to tell us these are things we need to identify and by using ATF I don't have to rely on my testers to have time available to do it I don't have to pay people to test and I don't have to worry about them Doing It Wrong by using ATF I'm ensuring that it it happens when I press a button and then it happens the exact same way every time as we get into ATF um a couple of key things that I want everybody to remember the first one is make a plan when we want to work with ATF uh it's really really important we have a plan on what we want to do we can't just go hey I want to run ATF I want to understand uh how detailed my testing is it going to be how much how in depth am I going to be so we're all on this we're on the same page throughout my organization about what we're testing we're not testing I want to know how we're going to maintain the tests it's really important as we get started in ATF to have a plan of how we're going to maintain these tests so if I just determine that there's an issue in Behavior that's changed who is going to determine this is the behavior we do want or behavior we don't want if it is what we want we have to go update those tests who's going to do that it's not something you can just set and forget we do need to have a plan of when something happens who is going to take responsibility for it so we're not scrambling at that point in time um as we go as we go through these couple again a couple other Bulls here but we just want to make sure we plan we know who is going to resolve those tests how we're going to maintain those tests when we're going to run those tests how detailed they're going to be next one is keeping it simple we don't in ATF we don't want to we don't need to test every possible permutation of what's coming out a human needs to create these tests and we want to make sure we're identifying the the lowest smallest thing we can identify and we want to test just to make sure it works does the flow firing do I get the right result I do not need to test is every field show up because it doesn't really matter it's it's not a big issue I don't have to test what servicenow ships what I need to test is the customizations that I've made and particularly if I'm adding new business rules new flows New Logic in there new Fields New required fields we want to make sure that that those things that we have changed that's what we're testing we just don't need to test everything else make sure we're going through that keep in mind execution time in ATF we we can test via the UI or we can test via the server side the server side is always faster but it depends what you're testing what you want to do so if I want to set something up I'm probably going to server side I'm going to create a record I'm going to create a user I'm just going to do that server side it's done sub second if I'm testing the UI which is if I'm doing browser compatibility testing I'm testing in the UI we want to make sure that we use the UI side for that it's important to think about it I can do virtually everything on the UI side but it takes longer and we have to make sure that the pages render and everything is downloaded and there's not a network issue that's causing a failure so it's important again to think about what's going on optimized for test speed if I can do it on the server let's do it on the server and if I'm testing the UI test on the UI I'm not testing the UI then don't test the UI make sure you use the the uh the server side they're all right next to each other there's nothing major about that but it's again just thinking about what are we trying to test and make sure we're testing that that falls in line with keeping it simple the last piece of this is we're going to make sure we're testing for outcomes and configuration this goes hand in hand with with keeping it simple but again what do I want it to do and does it do that then we're good I don't need to test every possible iteration of what comes in I don't need to test um check for all the different priorities if I'm looking you know does the drop down have all four priorities was that what you're testing yes then do that if you're not testing that then don't do that if I'm just testing that doesn't get routed appropriately does uh is the resolution code required before I close it test those things we want to have nice small fast tests that will identify quickly what's going on the longer my test gets the actual test the longer it takes to execute but also the longer it takes to debug to understand to identify ownership because there's so many moving parts so we want to make sure that we have just the just the um uh just the pieces we're going to test so when there's an error you know we're very clearly concisely who needs to look at it and what do I need to do um and next one piece about ATF on the on the way it works um the test Suites so with ATF we're gonna we have test steps that are that are pieces creating a user inserting a record clicking on a button those are all those test steps those are put together into tests and then those tests are put together in test Suites and this is really useful again to to understand what's happening what and how things roll out so I'm going to have my test steps we ship many many test steps out of the box you can also create custom test steps which are really useful for complex setups or just things we do really repeatedly so we have our test steps we'll put those together I'm gonna put them in a test inside of each test we will roll back when we're complete so I'm going to say I want to do these five steps these 10 steps these 20 steps when that's done it's going to roll back all those transactions and not leave a lot of Gunk around so one of the things I want to call out on this slide as we go through when I'm creating when I'm creating tests a lot of the things I'll do first is to create a user I'm going to create a user I'm going to give it some roles I'm not going to reuse users that are out there because that's going to cause me some problems at some point in time down the line a user changes department and now my test breaks that's not something that I want to have to deal with what we want to make sure that we're handling is that I'm going to create a user and the user is going to have this custom role so I know this user's there I know they're active I know who their manager is I know what groups they're in uh there I'm going to give them the roles that they want to have and they're going to do this one specific thing we just don't have to worry about different things going through so I'm if I'm trying to test a category or a group or something I'll probably create that as well unless it's something very specific to come through we'll create all the stuff we roll back so that way I don't have to worry about does this incident already exist it doesn't matter if there's a duplicate thing it doesn't matter because I'm going to create it it'll roll it back inside of each test we can pass data along so I'm going to create a user to the next test step I'm going to pass what was that user's ID it's going to create an incident to The Next Step I'm going to pass what the incident ID is so that I can move everything back again I roll back all that stuff goes away I can move past that data inside of a test but it's only with inside of a test because it gets rolled back but I'll put all those tests together it's part of a test suite and test Suites can be nested as well so I'm going to have a test Suite that says test everything I'm going to have a test Suite inside of that which is test it all of itsm inside of that I'm going to have a test Suite that says just test incident inside of that I might have one that says test um P1 incidents or test user experience for instance versus back-end business rules as I've been developing enhancements I'm making some changes I can just run a test Suite one of the smaller test Suites just test incident that's all we're changing or I can run the full Suite I can even go down and just say I want to run this one test as I'm going through I want to encourage my developers to be running these tests as they're developing I definitely want to be testing them as I'm moving through every change that goes through and then what we're talking about today with upgrades when I'm going through my major upgrades I'm going to run all my tests that I can come through but again don't run ATF twice a year let's run it every week and so that we always have these small changes to come through we have nice simple fast tests we can run them every night we can run them as they're developing we just don't have surprises when we come to upgrades nobody wants a surprise when we get to the upgrade ah and the other note that I'll put in here um the as we're running these tests we're generally going to create a user and impersonate ATF will handle that impersonation to come through so that these tests do run as those users and particularly when we get into the UI we want to understand what's going on so as I create tests I'm going to create tests for my for my fulfillers but for my for different end users as well with those roles to make sure that everybody is getting the right experience I might test the incident form multiple ways as an end user as a manager as a business stakeholder as a fulfiller ATF lets us do all those things without me having to engage with each of those people from doing manual testing either I have to manually impersonate them or I have to ask somebody from marketing to come do some testing I need to ask some of my fulfillers to take time off from doing real work to do this testing ATF allows me to simplify that I'm still going to use my users for acceptance testing to make sure they like it all I want them to do assess do you like this or do you not like this and do we need to address that not does it work ATF allows us to take out that question of does it work and a couple other tips that come through uh we we talked about these a little bit but one creating custom step configurations don't be afraid to do that we want to use the out of the box ones we have a lot of a lot of the out-of-the-box ones to insert a record uh create a user click on a button read a read a field but we can create new ones and particularly for test setups is where I'll use custom tests Uh custom step configurations if you're using um especially with some custom apps and we want to create some things to come through don't be afraid to use a custom step configuration look what's out there use what you can and then if it makes it simpler it makes it easier use a custom step configuration and again uh the custom sub configurations allows us to put whatever scripting we want in as well there is a run server-side script test uh step configuration where you can just paste in a script but if I'm going to reuse it let's make it more reusable by making it a custom step configuration um the the Run server side script step the out of the box one just by itself is something where I'm just going to do a one-off I need to do this one little thing one time outside of that we're going to promote it to being step configurations and I call this out because if you use the Run server side script by itself all the time you're going to end up with just a bunch of messes script around hard to maintain and particularly as I go through an upgrade maybe there's a change in Behavior if I have it as a step configuration it's reusable it's compo uh compartmentalized I can come in and update that one custom step configuration because maybe something behaves differently between San Diego and Utah I have to one place all my tests work again without having to go through every test individually create a user and impersonate again reusing is useful in some use cases but generally we want to make sure that everything can be tested in isolation so I'm going to create a user and impersonate that user with just the role I want so I don't have to worry about a user leaving changing roles group management changing you have to worry about all that because I'm creating and impersonating um the note here about updating if you update a record updating is always updating always works so make sure you validate um that the record did do what you wanted to do that's what how we test it's how we test flows how we test business rules we update a record and then we validate does it we query that record and validate does it is it giving you the right results is it now what do I expect um and then the record query really useful when I query a record because then I can start passing those variables to everything else so I do want my test to be somewhat Dynamic um but particularly with the users and uh with users that we want to create those users and roles because that does change quite often right the last piece I want to say for those specifically for those using ATF already and for those of you aren't this one slide is going to make this hour really useful for you really valuable for us and that's about the ATF test generator and Cloud Runner and we are going to do a demo of this next my favorite demo I ever do here um and the ATF test generator and Cloud Runner this is a store app so you're gonna have to go to the servicenow store um to to opt in for this and then install it on your instance this is compatible with Tokyo we'll I have this up in a second I think it's going to be patch four but you do need to be on Tokyo to get this if you're on San Diego you're not gonna be able to use this yet um but if you're on Tokyo you should get this and then going forward you can test generator and Cloud Runner it's two different things one app um the first is to generating tests and this generates regression tests for us so I talked about keeping it simple uh before that's true when a human's creating it but if I have a computer chain uh generating it for me then I don't really care and what the test generator is going to do is it's going to look at your data and it's going to create tests to fingerprint what your instance looks like right now we're going to fingerprint what our instance looks like right now before my upgrade I'm then going to upgrade and then I'll run the tests I'll run those tests again to identify if there's any changes in Behavior so the uh when we run this operationally what we're going to do is we're going to take a sub prod we're going to clone from prod to subprod it's really important that we have a fresh clone because we need data if there's no data or we have test data then our tests aren't going to be useful for us or anywhere near as useful as if we have real data so we're going to clone down I'm going to run the test generator it's going to look at my tables my Scopes whatever I want to look at or everything and it's going to fingerprint my look at my recent data generate what does my system look like today you know do that upgrade and I'll run it again it's not going to tell me that is are things good or bad but it will tell me these things behave differently these behave differently before and after the upgrade and then a human's going to have to look at that to understand is that what I want or not or what I don't want and again that might be hey yeah this fixed an issue that we had awesome or this Field's required now it wasn't required before but this is better awesome or this doesn't work right and we need to fix uh we need to fix the code before we deploy it this can generate thousands and thousands of tests for you saving you lots of hours uh we don't have to have a human go through and generate all this stuff this is going to complement everything else we already do it's not going to replace it it's going to complement give me that extra blanket of everything that goes through does not replace the functional testing that I created that we talked about and it doesn't replace using quick start tests and we didn't talk a lot about these before but with all the other box Solutions we provide tests to get you started for that will test out of the box material how things work out of the box I know for change they're 31 out of the box quick start tests each one of those you can run um separate you can run to go this is how it works out of the box everybody's made some changes so they may or may not work out of the box but you'll copy them modify them and it's going to help understand this is what the desired behavior is so whether you're creating the desired behavior is we're using the quick start test to identify the desired Behavior those are are awesome to test the business functionality and then regression test test generator is going to save you hours and hours and hours and give you a lot more confidence for only testing manually you probably don't have this level of confidence this is going to boost that confidence and make your upgrade smoother if you're using ATF this is going to boost your confidence as well but it's also just going to save you an amazing amount of time the other feature that we have on here is cloud Runner uh Cloud Runner uh is is awesome because it allows us to run these tests in the cloud before this if I'm running a UI test I have to have a client Runner running so if I'm running a UI test it's going to actually run on my laptop it will create users impersonate all that stuff but it is running on my laptop to interact with the server with Cloud Runner we can have this just run in the servicenow cloud and you don't have to worry about it you could do this before for those of you who aren't on Tokyo yet with our headless browser but you had to set up a Docker image you had to set up you had to provide a server for it you had to create a Docker image you could get the same basically the same functionality once we got into Tokyo and Cloud Runner you have to worry about it we'll host it for you and run those for you um awesome Time Savings allows us to fully automate everything so not only is the test running automatically but I can just schedule it to run every night and I'll get those results back test generator and Cloud Runner uh you can again the same app they run independently they're not you don't have to generate tests to use cloud Runner Cloud Runner will run however you want it to or any tests you've created whether it is a test you created manually whether it's a UI test server test whether it's from a quick start test or whether it's test generation Cloud Runner runs those all if I am generating tests these are generating ATF tests that you can go manually audit manually update as you want so again very independent but they're both great features that come with the cloud runner in Tokyo all right and now I want to do a quick demo for us of cloud Runner so I am going to switch what I'm sharing all right so we are uh in an instance uh this is actually Utah but again this this will work back to Tokyo I'm gonna go back to the servicenow store um so to get this is not on your instances by default you need to come into the uh servicenow store luckily I work for servicenow so I've already purchased this um this is free it's part of the platform um but you do have to come to the store and uh and uh opt-in to get it to it to agree to the the terms and conditions for it and you'll get it and again this is compatible with Utah and Tokyo patch 5 might have said patch four before but the latest version is patch five um this version did come out in May uh it came out uh last month to come through a couple small enhancements uh in here release notes are here um if you're on patch four not patch five well I guess we actually the old version was in patch two um but you want to be on patch five to get everything we need to get from here we'll see the release notes that that are that are there come here opt in then you go to your instance and install it you're gonna install this on your sub prods once we're there once we've had this once we have this installed I'm going to type in automated test I've already done that there is a one-time setup on the instance to select a cloud user so I'm going to come in and hopefully I haven't timed out well I was doing all my talking I'm going to select the cloud user and the cloud user is I am going to put a user in that has admin access I I actually just pick admin um what we just what we recommend for this is that you're going to create a system user with no password or we'll come in just type in some gibbage for a password nobody needs to know the password and then give that user admin this user is going to be used to generate your tests so we uh we do need a user here it needs to have admin access but it doesn't actually have to be system admin and again nobody's going to log in nobody needs to know the password there's certificate based authentication to to do all this but we do need to have a user to figure out which data we have access to so I'm going to set this up one time and submit it again one time per setup and then I'm going to go generate tests so I'm going to come down in ATF down at the bottom test generator this is the entire form I love this all you're going to do is put an email address in and say generate tests and not have to do that [Music] sorry about that okay uh so we're gonna generate uh whatever generate tests it is opening in a window I think you can't see so I'm going to there we go if you want to confirm that uh we can see this um okay so I'm hoping that we can all see the generate test we have a form and here we're going to type an email in that's all we need to have now uh as a note on subprods I generally have email turned off this will bypass those restrictions and it will send an email out this is uh because this is designed to run on a subprod it is designed to send me a notification when this is done this bypasses the normal email system so it will send an email even if you have email disabled on that instance and all we have to do is Click submit it's going to generate tests so all you have to do I'm gonna go in advance so we're going to take a look at what's in here um we have a few options so the uh we can generate for everything which is the default all Scopes all tables I can also come in and select a specific scope um so if I want to just generate something for um this may not have anything in here but I can generate just for a specific scope this is great for my custom development because I can generate my tests for those Scopes I'm going to say all Scopes you can also choose a table if you want to test just a specific table you can select that table how many tests so this is set to create uh 10 000 tests or less 999 tests and no more than 50 tests per table that's going to be thorough enough and send my email so I'm going to actually uncheck that I just want to create everything and I'm going to click submit generating tests can take several hours up to sub up to a few days to generate um the latest update did have some performance improvements in there but it still will take some time don't expect this to happen within minutes it will take hours and possibly days it's going to be doing a lot of analysis on your data if I'm just generating for one table probably pretty quick but if I'm generating for thousands of tables on my instance it's got a lot of work to do to profile what you have to generate those tests and to to validate those tests work correctly uh when we generate tests so it does take some time but I like to think of these tests as disposable tests they're not things we're going to keep forever so the quick start tests I modify the tests I create manually those I'm going to keep and run and maintain generated tests are I'm going to clone to a subprod generate my tests run them for that upgrade and then I'll probably throw them away I'll probably delete them I might keep them for a little bit of time but they are all based off the actual data and they will get stale and deviate as we make other changes coming through the there's no cost to generate tests it's just some time so it's something we need to plan for um if I'm going through my upgrade probably the in my upgrade cycle I'm going to early on I'm going to generate my tests and keep those for a few weeks we need to plan for that time so again not expected to be minutes but hours into up to a couple days but there's no real cost right I just have to press a button and be gone for the weekend and I'll have them so make sure that you're not afraid to regenerate those tests we're gonna go look at the tests that are generated a question came in uh about performance so you shouldn't notice it as it's going on it's only using one semi four so it's one user who's looking at a lot of data but you shouldn't notice it again this is going to be on a sub prod to come through it is it is working but it's not going to be really anything you notice when we run tests in ATF as well it's like a user's doing it um so if I'm running one session it shouldn't impact anything um it's working it takes time but again it's it's controlled so you're not really going to notice anything you certainly can do other testing that's going on while I'm generating you can um you could do other testing on your testing since it's not going to bog it down completely but it is it is doing some crunching in the background all right so my tests that have been generated we're gonna go pull out them uh pull them up they're going to be named ATF generated Suites and as we come through in this case I do only have um test data so my tests are not going to be good but as I come through let's look at uh this assessment category results super interesting because I don't even know what's in there but I looked at my data and it created a bunch of things now when I create my test manually I we talked about this a lot I'm going to create users and impersonate because this is looking at real data it's always using real User it's never going to create a user to come through and I'm going to switch to Globe and so what we have in here is I have my test it generated the test with the table name and the user and the view it's looking at it's going to impersonate that open the form um it's going to make sure things work at it that everything loads it's going to set some values and go through and again this is this is all going to be UI based or the vast majority will be UI based so it it's got some timeout checks that come through it's going to make sure everything works very detailed this only has eight steps so we can get to a more interesting one it's not a bad number if I was manually creating it but if I'm going to go look at incident you will see some tests that are generated that are hundreds of steps hundreds of steps to come through because it is going to test every everything it can as it looks through and you can see a level of detail if I'm manually creating this there's no way I'm going to go to this level of detail I'm going to check what's what's there what's really important to me and not all these things are really really important but when I have the generator do it it's going to test everything why not right again we want to fingerprint what the system looks like before and after those upgrades so lots and lots of details it's checking is everything there and going through again I didn't have to do any of this it just generated for me um and I want to I want to pull up uh quickly what the um Cloud owner looks like and what you have to do for that so we do have to set up that cloud user which already did for the test generator and if I'm going to run the test all I have to do is say I want to run in the cloud Runner so I have a choice do I want to start a test Runner which is local if I'm doing cross browser testing I'm going to be using a local test Runner this is what I was able to do before you're able to do this in San Diego once we got to Tokyo we'll have Cloud Runner um you do have to install this for the to work I think if you don't have it installed it'll actually give you the option and say you want to go install it but once we have this installed we click Cloud Runner and we'll click run test and then it's just going to go right in the cloud it actually looks the same in this interface whether I have a a local test Runner running or has the cloud I don't know if this is actually connected right now to to do it it's trying to connect into the cloud Runner run those tests and give me the results the results also are stored the same there's no difference it's just did I run it locally or uh did I run it locally did somebody else run it locally did it on the cloud you'll be able to review those results in those test results I'm actually going to cancel this right now that'll run through for those of you who haven't seen ATF we're actually going to run this locally too so I'm going to come in and click run test come through we're going to run test and I'm going to run I'm going to start a new test Runner so I'm going to run this locally it's going to open up a new tab and it's going to run those tests for me I'm actually going to see it run as it's going you'll see it coming in running the doing the test step waiting making sure everything loads through and it's going to be taking screenshots as we're going to there's an option whether you want to keep those screenshots for everything or you just want to keep those screenshots when there's failures I think the default is only when there's failures that's what I do so if everything works fine it just says it's past it I'm good but if it doesn't look right if it doesn't pass it's going to take a screenshot so I can review the failures go that's what went wrong and then go from there and then I can again decide that's what I want that's not what I wanted so it's going through right now it's running those steps if we come back to the other tab this is what we'll see with Cloud Runner as well it's running these things in the background as it's going through when tests are running if you're running them locally don't do anything else on the instance from you from this session because this session is what's going to roll back so if you did something it actually would get rolled back as part of the test it doesn't know what's part of the test and what's not it knows the test is going on when it's done with the testing rolls back if I'm doing this in Cloud Runner that's fine I can do other things if I have it in a different browser it's fine too but this session is part of the test so this is running it's running through everything it needs to go through and you'll see it mixing server side UI side and server side it knows the difference so we can optimize those things and I'm going back here again it's running those steps to come back I could watch it run but again I don't need to and let's look here so it went through my UI side we came back to the server side and if you can see the difference it took a minute to run 10 server side or a client-side UI steps and then server side it did 20 more of those in a second I there's a big difference to come through and then we did that rollback we'll go to the results and what we're seeing is this is everything I wanted to do and they successfully did all these things if I had a failure it would tell me where it failed and stop and I can go drill into what's going on allow me to recreate it with the with the test regression steps it's good we do expect these all to pass if I'm testing specific functionality again the simpler it is the more I know who should go look at it and where that issue is all right uh so we went through the test with the demo we went through it there um and let's go back to the deck I'm gonna take a second to answer some questions all right so a couple questions that that uh came in in general having a deal with ATF okay so we have uh where do you build ATF and where do you run ATF so we're going to uh build and run ATF on our sub prods um an important note though with those tests that we create I'm generally going to promote those to production um and they're going to go with the app that it's going with so I want my tests to go to production when I build them just like I build any if I build a business rule I'm going to build a test that goes along with it I'm going to promote all of that through the app repo into production and that way when I clone I still have my tests when I run tests there is a persistent property on the instance which says um can I run tests in production or can I run tests here the default in production is that you do not run ATF tests the system will just not let them run once we get into a subproduct we'll let them run so you only want to run ATF tests on a subprod again that that rollback would be could be bad in production so we want to make sure that we um that we're testing in a subprod but we are going to move those tests up the same thing with the store app um you can install you can install the store app in prod that's fine but we're not going to generate tests and prod um yeah probably not gonna do that we're doing the code and generate them there so the tests that I build the quick start test that I modified those will go into prod the generated test probably won't go to prod because I'm going to generate generate them and throw them away when I'm done um but we're going to always run in a sub prod um when should you not use atfs um if ATF doesn't test what you're looking for then you're going to use something else ATF doesn't do mobile ATF right now doesn't do configurable workspaces so we need to use something else to help test that but anything I can use ATF for I want to it's free it's easy we want to get through that um and the the last piece that I thought with ATF uh people ask me how do I get started with ATF and the way you could start with ATF is you just get started you write a test we start from now going forward I don't worry about my test coverage going backwards but if I start every change I'm starting to put in I'm going to start with my tests we're going to use the test generator and start getting used to running it and creating those regression tests and running them and generating them and running them and generating them and those business use cases I'm going to use Quick Start tests for the stuff that's out of the box and get started there like the name says very quickly and I'm every time I make a change I'm going to start requiring ATF tests customers that are successful with ATF have part of their definition of done they're part of their acceptance of a story is that you have an ATF test the test doesn't work and then it does not work start building out your library it will take some time but you're going to sleep better upgrades are going to be easier as we go forward um a couple other quick things before smile tells me that I'm running over uh it does not need to be system system admin uh to as your Cloud user but it does need to have admin access the reinstalling if you install in prod then when you clone you won't have to worry about it or you can reinstall it as it goes through and customization is really important one for the test generation the test generator doesn't know what's custom and what's not it just is going to look at fields and look at your data and figure it out so whether it's customized or out of the box it doesn't matter it doesn't know I don't have an option to say don't test customizations it's going to test them if you have those fields you have it on your forms you have data in there it's going to test it um and as far as a Timeless generating for all scopes for everything uh it it's all dependent on how much data you have because it's the generator has to look at your data and profile it to figure out what what the common values are on what we should test so limiting the total number of tests again the default is 10 000 and 50 tests per table um it will stop there but you can you can adjust those down or you could adjust them up as you want to and how much time uh again we just thought we have some important performance improvements that came out in May but do expect hours to a couple days but it will be very very much data dependent okay I'm gonna rapidly talk about some other more rapidly than I normally talk talk about some other things I'm getting started with ATF in the community there is a center there's a getting started with ATF uh Center of expertise in here has links to several webinars that talk about ATF in a lot more detail how do I get started what are my best practices um and then a lot of resources uh now learning training some Labs from uh from knowledge lots of great content in your library place to ask questions um and you that's in the uh in the chat as well other piece I want to talk about was instant scan and I think I have about two minutes to do this so instant scan is a great complement to ATF so it's going to identify um issues in your configuration your patterns that come through it's going to help you enforce best practices you can write your own checks uh for instance scan to make sure that your developers are writing code to meet your standards and it's going to allow you to uphold those quality standards as we go forward we're going to go through um things uh see if I have the example there I'm gonna actually go through this a little bit faster but again there's a lot of out of the box checks that come out of there instant security Center uses this it's going to check for um things that you want it to be so whether I'm running a table query I'm running a script it's going to make sure that every bit of code that you're releasing it's your standards and meets the best standards that we have out there and what this allows us to do is to validate so we all develop we all develop already so there's plenty of things to help us develop um we talked about ATF and how to test does it work but this allows us to validate with a static code analysis about are things working so if I want to look for um there's no the old one we must have had no package calls in a script or I want to not I want to make sure that um uh ACLS don't have Glide record in them I can create a check that says well anything on sys security ACL does not have the string Glide record in it um whatever I want to look for lots and lots of examples here that you can go through and make sure that that best practices for servicenow and best practices for you are followed we might have different sets for instance scan Suites that are run for General Health we have one that's set for security if you get the instance troubleshooter from support uh that's from the store that uses Instant scan as well and again you can do it if you have app engine you have app engine Studio when you go through the deployment process it's actually going to run an instant scan check that you can inject your instant scan checks in there lots of great content in here not to test those at work but to go is it configured correctly is it meeting my best practices as an example this is one that comes through where I'm looking for that my does my scoped app contain any uh any task extensions is it something I really want to be aware of that we have some additional scrutiny for this uh this is out of the box is part of one of the ones that the pipeline works for 80 for AES but it's doing a very simple check looking at this table is this exist and flagging those things and then what we see over um here is it's not just saying hey this happened or a count of these things it's going to give you those specific records so as an admin I want to make sure I'm running instant scan before deployment to go is Everything is Everything up to my standards and if not I'm going to go manually look at it it's going to tell me the specific records and the specific issue I need to go look for I don't want to do manual code reviews I want to use instant scan to do those code reviews for me as much as possible we can run through this look at those look at any issues we have and then go from there awesome asset something we definitely want to leverage an AES audit when you go through deployment is automatically going to write ATF and instance scan make sure we're configuring those putting in the right test putting in the right instant scan checks for us to make sure that we can sleep easy at night and again every time I'm doing this deployment I'm either going to make my upgrade easier or harder let's make it easier and since again at ETF are going to help us do that last piece is upgrade plans upgrade plans came out in Tokyo so if you're if you're in Tokyo and you're going to Utah you can do this the upgrade plans themselves um very simple uh that comes through I'm not doing a demo because you need two instances to do it but in the past as an admin when I'm doing my upgrade I would go through figure out my scripts figure out all the manual steps I want to do now I'm able to use an upgrade plan so on the first instance I upgrade I'm going to create an upgrade plan it's going to be my Builder and I'm going to say these are the skips I want this is what I want to do I take that creates an app pushes it into the app repo and then I can apply that to all my subprods all the way up to prod and test is everything going well and it gets rid of the a lot of the human chances for screwing things up it makes it simply faster to do and reduces error upgrade plans are awesome so again if you're on Tokyo take a look using an upgrade plan to help you go to Utah it doesn't it works great for major upgrades but also if I'm doing my uh mine along the way we're able to push things through a question came in about app repost so it uses app review by default I don't know if you can just apply as an update set I think it does need to use app repo um but I will have to go look there we can look in the docs to see if you can use it without the operator the default is to do that repo because it's just really easy to push things through um and uh as far as upgrade plans go I'll get my plug-in a second to come through more detail about using them um I believe we have an academy session on it and there is information in the documentation um with that actually going to talk about it for me about the uh academies I'm going to hand it back over to Shamel to give us a bunch more resources about what we're talking about what we talked about today and some other things to help you in your next episode thank you Adam alrighty I'm going to go through this pretty quick because we're almost at time I want to make sure that we can answer some of your questions um and Adam will be in the chat answering some of those as I go through these resources first and foremost I want to make sure you know about the Utah release upgrade kit this is available on the customer success Center there's gonna be a link in the chat the upgrade kit will give you a full summary of all of the latest Innovations from the release and as well as a playlist of valuable links demos blogs and all the resources that you need for your next upgrade and what I love about the upgrade kit is it's available on both PDF and PowerPoint so you can customize the presentation when delivering to your stakeholders I also want to make sure you know about the now create upgrade methodology this gives you step-by-step guidance on upgrading your instance you can use this um if you already have an upgrade plan you can cross-reference this with your current plan each each stage has an in-depth list of resources demos Etc lastly I want to make sure you know about the servicenow upgrades and releases Community Forum this is a place to ask questions and get answers not only from service now experts but from your peers as well you can get the latest best practices resources and you can sign up for upcoming events just like this so this event uh today was a part of the live on servicenow series which is an integrated event series to support upgrades adoption and deployment of servicenow products and as Adam mentioned he leaves the platform Academy series um which happens bi-weekly um so we'll have links in the chat for both I'm going to skip past these key takeaways so we can get some of your questions all the links that we share today all the resources we share today will be linked in the chat alrighty Adam are there some questions that you want to make sure we answer before we wrap um I think I got through most of them on putting some more in the Q a uh anything comes through um as far as next experience so that is on the roadmap right now uh to to come through I believe it is not in Vancouver that it will work um it does uh next experience it does work in the unified nav if you turned on next experience it works all the listen forms uh still work but the configurable workspace doesn't work so it's part of net experience does work you don't need to be afraid to turn on next experience but the workspaces themselves uh that ATF doesn't support those yet and that was it awesome okay um and then we've got some questions on Where can I access the recording for this session we've we've put a link there in the chat um and after this session is over you'll also get a link email to you um and actually I will put in one uh there's a question that came in about uh walk-up experience in ATF tests so the quick start tests which you definitely want to be looking at um every app I believe ships with them um so I don't know if it doesn't and and um we don't cover workspaces we do cover service portal to come through uh once you install the app some of the apps do ship with the ATF test put inside of them um other ones there's additional plugins or additional apps to come through so make sure that you go into system applications and go look and see if they're uh there are ATF tests for whatever app you're going through and there are um six or seven hundred actually going back a couple releases there were six or seven hundred quick start tests out of there lots and lots of stuff out there that's just waiting for you to take advantage of so make sure you go look for those plugins look for those apps that contain the tests for the apps you're already using wonderful well with that we will wrap thank you so much Adam and thank you all for joining us today um if you have any other follow-up questions like I said you'll get a link after today's session with the recording there's also an email that you can reach out to But be sure to check us out on the upgrade and releases Community thank you all and have a wonderful rest of your day