[Music] good morning good afternoon good evening wherever you are and where you are welcome to creator toolbox this is the series that brings you all of those nifty servicenow developer tools that you can get a hold of and we're here to bring you the latest on san diego where's that badge earl left shoulder there it is that tells you this is san diego content can't wait till it starts it says tokyo in a few months so looking forward to it my name is chuck tomasi i am a senior developer advocate at servicenow been here for a long time we'll just say that and was a customer for a bit before that and a whole bunch of decades of developer experience and uh over there is the mighty earl hi everyone i am earl duque i am a developer advocate here at servicenow not quite as been around the ecosystem as long as chuck but uh before that was in higher education and before i did servicenow i was in student affairs so that's what i've never said on air before so that's a little bit more of my background where i got started but yeah i'm here to do color commentary for chuck's creator toolbox today and he's driving the comments and everything else behind the scenes so thank you very much today earl uh quick reminder i mentioned this is part of our san diego series of content we've got a rolling calendar of things still more to come we've been doing this for about six weeks maybe seven i lost count started on february third whatever that makes the math work out to you can find our index oh plenty at that link on the screen devlink.sn san diego now we're just with the cameras it is it's right there below my feet and i invite you to go over there if you've got something on your uh personal interest list check it out mark your calendar if it's still coming up or if you're watching this in the future go back and see what you might have missed there's a lot of good stuff that's happening there from flow designer integration hub we'll be doing a little bit of automated test framework today we've got rpa still coming in the future girls earl's nailing down some of the buttons and and zippers on that thing so we'll get that all tightened up we've got as i mentioned automated test framework today friday's live code happy hour is more platform loveliness we're going to be implementing the new facebook open id connect so you can use facebook as an sso i learned a few things about that late last night well i had our developer online from india i went hey what's going on here i've got a question so there's always something new to discover isn't there earl yeah do you remember what i covered on tech now back on february 3rd you'll remember this we've got a way to insert break points and i'll discuss those as we get to them and a way to pause the test before the rollback at the end if you remember automated test framework as a quick background is an in-platform test to do your functional testing why is this important well you want to make sure you have quality releases of content before you release something to production so you build your test in your dev environment and then you test it in qa if anything gets wrong you go back to dev you can do this very rapid cycle build build your test as you're doing your development and this isn't just specific to scoped applications you can do this on global applications you could do this on plug-ins you could do this on anything that you want to test to make sure that it works the way you expect before it goes into production and those same tests can be rerun before and after an upgrade for example run it before an upgrade to make sure everything is working as you expect and then after the upgrade to make sure everything is working as you expect because it doesn't do you any good just to run a test after an upgrade the upgrade may not be at fault it may not have been working before so quick pointer in there a lot of people go yay we'll run atf after we upgrade if you didn't run it before you have no baseline of which to base it on and it may not be the upgrade's fault that something doesn't work so just a little best practice there for you so that's atf in a nutshell it saves you from learning a third party tool putting it on the platform customizing it on every release or even worst case handing a spreadsheet to a an intern or a summer student and saying here run these and hoping you get back meaningful data i've run many a manual tests in my day and uh sometimes people click on things that they shouldn't be clicking on it's like why did you do that it wasn't in the test or they give you back test results and say i got an error message like what was the error message what did you do would you say it it's just horrific so this gives you that in context way of saying this worked or this didn't so let me get started and show you where automated test framework is this is our next experience if you're if you're new to the video and you haven't seen this yet the menus look a little different but should still look very familiar enough that you know where we're going so i'm going to go to automated test framework we've got our whole wonderful list of stuff for that application i'm not going to go through a whole build out i've already built out a test my my use case in this one is an application we're prototyping for creatorcon this is just a simple no code application that has student approvals for like field trips so the teacher creates an activity the activity gets a student and when the student is added to the related list a flow kicks in and says we need an approval for this student to go on this activity whether it's the dairy farm the sports facility or the correctional facility i don't know what teacher would take their student to a correctional facility must be a really interesting course if that's where we're going did we just get some insight onto your field trips when you were younger chuck i didn't go to any correctional facilities making up demo data is one of the hardest things we do as developers am i right and sometimes my imagination needs a little alcohol fueled inspiration a distillery we can take a trip to the distillery of the kids don't get me started two new things i want to point out on this uh one is if you look at these ui actions see if i can do my pinch zoom trick there it is we've got run test which has always been there but we also have this new button called debug test in san diego the rest of this is pretty much the same okay we also have a new related list here called breakpoints i'm not turning this into a drinking game for the podcast okay just leave it at that and my test is as usual we've got some client-side things where we're opening a form we've got some server-side things we're doing record queries not a horrifically long test but it's a test nonetheless let's go through a typical day pre-san diego and we'll run the test and it says i need a new client test runner because there are client bits in here starts me up and away we go i love watching this part this one's going to run a little slow i forgot to turn off uh the uh picture taking part where it grabs the screenshots so it runs and i mean obviously if you're observant and you're watching something other than that status bar you noticed there was something horrifically wrong with that okay change screens and it's a that's not a good screen you want to come home to yeah so you go okay something failed in the order of three to five let's go to our result so we click go to result and we get down here and it says well the first couple passed i created a user which is often what you want to do don't use an out-of-box user don't use one of your regular users because people change roles and your data changes this is a way to make sure that i statically use this school principle is the cis user account that i made as my test user then i can grant them the roles etc etc okay but i got down here unable to set field field start time is not present on the form interesting i jump into that test i don't have any attachments let's see oh i have some attachments up here let's take a look two attachments and the way i would do that is look at the image open the image and that appeared on a whole different screen let me move that into frame and earl what's wrong with this record it's right the start time doesn't appear on here there's nothing on there that looks like there's nothing on there it looks like an access restriction that's what i'm thinking so let's go back to history test oh not test results i want the test i love that history menu when a when a screen loads and there's nothing on it that usually means there's an acl restriction problem something is happening around test three so what i'm going to use is our new breakpoint this is so easy you right click or what do they call that alt click something and we have a ui action down there called add remove breakpoint there we go don't take a drink and it indicates this with the red okay if you're using the old style of via red dot if you're using a new style it'll be this way so just some indicator on there says you've got a break point also there on the breakpoints related list we have one record and what's not shown here is that this is my record so if earl were in here testing at the same time his break points would not interfere with mine these are specific that's nice i don't know why it's not on the related list but maybe is it filtered to me no okay well whatever let's use the debug test now when i hit debug test you'll also notice that we have this new checkbox pause before rollback we'll get to that in a second if checked the test will run the test will pause after the test is completed but before it's rolled back think of that as like a break point after the last one if you will okay if i put a break point here it's going to pause before it gets to that step maybe we just leave that there for now all right let's go ahead and hit debug test again i'm not going to pause before rollback but i'll run the test and we are going to see it gets up to that step now there's two things in here that i want to point out there is continue and step over continue says great i'm happy you paused now when i hit continue run to the end or run to the next break point i i've seen what i need to see this gives you a chance to go look at logs go look at records as they're being opened etc etc now remember as the test is running i am impersonating whoever that other person is so you may not have full access to your system atf really takes over and says do an impersonation of that test user that was created so step one is create me a school principle step two is impersonate the school principle step three is let's go open the form step over is a way to walk through the test steps one by one in my control it's like a manual advance one step advance one step advance one step now we are running a client step right now so this really doesn't do me good to do some introspection what i need is over here and i have the same two controls on the right hand side continue and step over so it's paused right before step three open form you can see i don't have an open form so let's hit step over and it's going to try to do step three open the form the other thing i want to point out over on this side is you see on the left it says paused at break point 10 minutes remaining if you pause too long it will just say hey you must have fallen asleep i can't hold the system in this stasis forever so we're going to move on so there now it's nine oh i've been talking too long that didn't feel like 60 seconds to me but we'll take your word for it now at this point though because especially for the sake of if someone forgets that they had break points it's creating data so you need to the system wants to clean it up and get to the end end of it right wants to clean it up and you know you're you're holding some system processes hostage at this point too so probably not the the best thing to do forever but i've got full control over where i'm going and how i'm doing this and i can see at this point there is clearly an access record an access problem here now how i choose to debug that i could run the test to completion go okay it failed i think i know where the problem is it's in this step so clearly the school principal doesn't have access to this record so just for fun let's continue it's going to error out it skips all those other results i'm going to get the same looking you know red blue red yellow you're like chuck that what did that prove it proved that i had control long enough to see what was going on because if i wasn't looking at the client test runner wouldn't have noticed so i know there's a problem with step three yeah something to notice too is your other break point didn't hit because it had failed earlier so there wasn't another place to go to right if i go to the results it says hey you had a failure skip skip skip so it's not going to yeah it it died and got out of there so let's go look at the test let's go look at the test itself i think i know what the problem is it what did you say it feels like a permission issue okay so if i am back to being me i'm no longer impersonated it did stop the impersonation um you know i don't see the impersonate user here but we're gonna interesting i thought i had an impersonate user step in there not gonna worry about that right now uh does the create user have an option to immediately i think you're right i think it is uh impersonate this user as a checkbox and lookie here there's no roles on this user um that could very well be the problem and if i were to go and look at the table let's let's go into studio for example look at this app yeah it sounds it sounds like a access to create record but no access to any of the fields access control says create needs ah you need to be a student o admin um and i'll bet the same thing is true for read and write which is why nothing showed up on that form sure enough i have to assign a role to that user so in this create user let's go roles x under student o admin done lock it down update it since we're debugging too do you want to check your second create user and make sure that is that one's not getting impersonated good point not impersonated all right we're good and because i'm not impersonating this they probably don't need a role it's just sample data that i want to put in a field later right we need a teacher on this activity record and i said all right i'm not going to use an existing teacher because that again that data may not be available every time i run the test could be different in dev than it is in prod so i like that it shows any assumptions i appreciate that it shows the with roles on the description right there oh there you go that would have been helpful i should have noticed the absence of that but i'm not that experienced with atf so maybe i don't recognize it let's do a debug test again run test it's going to run up to the client spot until i switch screens go switch screens and hit this paused right before that test notice again notice on the left paused at breakpoint i'm suspicious of that counter right now let's hit step over so we're going to advance this one step yay i've got the form that step simply said open the form okay next step is populate some stuff with the teacher that was in step two with the activity that was come on i think there's a i'm at another break point i'm at another break point oh you're stepping over keep going and oh we have another error look at this going step by step activity is a mandatory field we can't submit this in effect it just it's gonna die again right so let's continue and once again i think we can put away studio we've done everything we need to oh look it's still broken i have two problems in this thing at least two problems with this test who wrote this i need to speak to that author let's go find out what that is i suspect it's gonna tell me the same thing uh save action was aborted before being sent to the server now i wouldn't really infer that i'm missing a mandatory field from that error message so again walking through step by step shows me what i need to do your first question may be why don't you just put a break point at the beginning and then step by step through the whole thing till it works you could do that you could do that i like to uh you know put it where i suspect the error is first or right before an error happens because i i'd rather not get down to a 19-step test and go one two three four still working you know that makes for slower debugging if you have to step through successful things every time so we know we have a problem here and i could see from the screenshot that there is a missing mandatory field which means let's go back to the test and somewhere we could actually remove that break point because we know that step works now and let's try it on set field values blink it's got to redraw the form so it's updated and i think yeah we can see it from there it's missing that field it is missing the activity field yeah maybe when this test was originally written activity wasn't mandatory don't know which is which is a good uh use case that you're demonstrating for real world why these tests are useful because what you might have expected as a passing test before has changed over time yep yep okay i'm not going to use a data pill i'm just going to say we're going on a brewery tour as every third grade should and let's do another debug test you can see these test results are piling up too in the test results related list so that's a fun thing debug test run test let's go over to our client test runner so you do have break point still so it should stop yes i put i moved it from step three to step four and we're at a break point right before step four so open the record now i'm going to step teacher brewery tour start time end time everything looks good mandatory fields are filled out it's on the next step and we'll go to the next step you don't have to go step by step here i could say let's continue so it says let's save the form now it's going to attempt to create a related list record which is all going to be server side and from here i might be able to go continue back yeah you should have a couple more client steps to do yep oh yeah there's a couple client steps at the end that is due so it's filling out this approval record then it's going to approve the record hooray take a screenshot i should have taken the screenshots out but it's been approved the permission is approved and pause that break point remember i put a break point right before the very very end at this point again i could go around and look at system logs maybe look at these activity records say hey what's in sis approval approver got list i've got a permission record i can see this stuff actually happened and is being done and you're still impersonating the person because you could tell with the security risk constraints oh yeah if it were me i'd see all of those but this is before the last step what was our last step i think it was a um [Music] a validation or an assertion of some type history test oh not test step wrong one test you're not gonna be able to see it cause you're impersonating you are right well that's a big bummer okay we're gonna have to finish this uh so i'm before the last step let's just hit continue one more time and it will finish things up so that gives you an idea of how breakpoints work and let's refresh this screen now that i'm back to being me test results success i have success success there's something in here that i wanted to show you in the test steps that last step here we know this worked so let's take that off easy as that just adding and removing breakpoints and field values validation um i have no assertion on this validate that the form matches these fields i thought there was a oh the assertion was on the ui action never mind i was looking at the wrong one there's a submit where is it click a ui action when you go into this client actionable thing to click a ui action uh you tell it what ui action you're looking for i'm saying i want to click the approve ui action and then there's an assertion type and i love this annotation right above it right now it's just going to click it and not care but i may want to make sure that the data was actually submitted to the server you might have a modal that pops up you might have a reason why you hit a uh set abort action and it wasn't actually submitted so there's there's different ways you can do this you could say was it cancelled in the browser or was the page reloaded or redirected this is actually the default if i remember right form is submitted to the server why somebody changed it i don't know you're saying it changed to none as the default i changed it earlier today i thought it was going gonna give me an error but it didn't okay let's remove all the break points but this time when i run it i'm going to do debug test and check that pause before roll back both of these are suggestions that people made through the idea portal so if you go to the community community.servicenow.com and you go to the top there's an idea portal where you can enter in new ideas for the products or enhancements or possibly bugs and then upvote what other people have said and the break point and the pause before rollback were included in these suggestions and we said yeah we can do that i like the pause before rollback idea that someone suggested it's so helpful for that sake of uh verifying if there was a false positive because sometimes you just want to let it run right and then look at the data at the very end and do a quick eye glance through or make sure that everything looks appropriate before it actually you say like okay everything was good exactly i like to look at the records and just give them a visual inspection say did it do do i need to make my test better maybe i need to fill out another field or maybe a business rule ran or a flow ran that i'm not seeing because when it does a rollback it rolls back i mean it it it puts everything back to the state that it was in before so this is done what happened to my pause oh wait it's called the status bar isn't that the very end where's my indicator that says i'm pausing before rollback actually says waiting for a test run so rolling back db changes here we go right here rolling back db changes that occurred before the text executor so it's not doing that step for me it's oh there yeah so the your browser client that um controls the atf is already released but the actual role that doesn't have to happen in that thing so only you have to continue in the step over here still i got it it was a bit hidden watch for this message here okay and then down below you're also going to see here i've gotten again 10 minutes to do something nine now but at this point i could go well i want to see that activity table and since my impersonated user now has oops student permission access you can see ah there's 29 yep we're going on a brewery tour there's the student that was added there's their permission it was approved you can go looking through here you could potentially go looking through system logs right see what's happening not much but uh you know a few things happening with the test runner that don't interest me in case you have gs info statements in some of your scripts whatever that's all going to be there so very nice way to do that now earl yep how do i continue if i'm not that person i go history test oh the impersonation is over it's just waiting to roll back so i don't have to worry about getting back to this there was oh where was it i saw something somewhere where i was on a test results screen or test and it said um you know return to test or continue test while i was in that impersonation state i wish i could remember where i saw that i don't think it was here oh because i needed to get back to one of these screens and i went wait but i'm not heading in and i don't have access and um so this is this is the previous one that ran to success we haven't finished until we've done the rollback so at this point oh this is what it was i lost my dialogue i lost my modal how do i tell it to keep going opening can you open the test result again can't open the test results they haven't been logged yeah i'm right where i want to be so how do i how do i continue and i thought it was it wasn't there uh if i do debug test again it's going to launch another one right manage your test runners here is it going to start well oh waiting while another one view the test in progress there it is see that you just have to be able to read yeah can you run another test you're already running a test so this one is paused if i go in here it says show progress now i'm back to where i was so if you happen to navigate away from the test runner the test as it's running you can still get there how many minutes do i have left six i knew i saw that somewhere when i was learning about this last week i think before you press the button my my little window that you know a lot of times what i i know people will do hey look it came back uh a lot of times what people will do is they'll open up a list of records like hey i want to look at the activity list open it in another window so you do the you know the shift control thing or shift command or and it opens up that list in another window i'm getting a little lazy on that now that we have the next experience where everything's in full screen anyway are we done are we done go to result there's our step results there's our test results blah blah blah blah blah blah oh you're re-impersonated somehow no i don't think i reimpersonated i think the screen is just a little what i think you're because you remember how you were trying to oh i am i remember school principal you you were running a test again oh as the school principal i ran it you're right what world am i in right now brewery less in progress more info how did the school principal launch one no i don't want to go to the docs page thanks yeah apparently i am can i end impersonation okay that was interesting i ran a test within a test and became that other person again i'm curious to see what the history will say on this [Laughter] todd in the chat uh mentioned something that i usually do too when i run atf tests and i usually run these tests in an incognito window to avoid impersonating users while i that's a good idea okay this one is still running let's show progress does am i it's waiting for me to get over to the client runner i'll bet yep yeah yep yep okay so i have to finish the second test before i can look at the results of the first test and it's just because you press the um you press the run test button which got you back to the dial dialog window yeah i think before you press the run test button there was a there was a blue link that said view uh manage test runners that you could click on instead of hitting the actual run button let's close that window it's messing me up okay this one i would expect to fail because it was launched by somebody else the student runner step results success success success skipped okay not worried about that particular run i want the one before it there we go so we had a success on the one that paused before rollback that was not the best demo of pause before rollback should we just do that again real quick sure okay i am me i'm back to being chuck multiple personality syndrome on an instance it's a lot of fun i've got my tests i don't have any break points i'm simply going to where's my run test you got to refresh oh what you know what i'm going to log out and knock back in i don't know what state i'm in anymore that was clearly a mess okay history test that sounds like something i had in high school a history test no break points and break points are persistent if i were to log out and log back in these records would still be there which would be represented on this related list so i don't need to worry about that let's debug test start a new test runner or i could use the one that's already there let's use the one that's already no no i don't trust this one let's do a new one i wouldn't do that i'll try pause before rollback that was part of that what the heck client runner runs activity record gets created teacher field activity field there they are start time end time and goes and does some server stuff looks up the permission record that got created gets the approval off of that now we go back into the approval record and approve it so not everything needs to be done on the client side there's a certain beauty to doing it on the client opening forms and setting field values because you can actually see what a real user would do but querying for records needs to be done on the server side and some of these things were done by a flow in the background there are timers on some of these that say hey wait five seconds or wait up to five seconds or 10 seconds or 30 seconds if you know process is going to happen otherwise your your test step may come along and say i need to find this record but the flow hasn't finished running right thank you to lisa for helping me with that little scenario okay test has run i'm waiting before rollback here's that not done with the rollback thing so the question is earl am i still impersonating according to this it seems so but it did release the runner so let's do this let's go to all i i don't think you're impersonating anymore student student permissions let's open this up in a new browser tab lisa says you're welcome by the way good all right and i can mess around with this record it's still there it's still alive it's not going to be rolled back interestingly enough if you've been watching the numbering sequencing they are going up so the number is not being rolled back and i think system logs are also unaffected it doesn't roll back all the records it just rolls back the ones that you're trying to test uh let's save and then we can say who updated it with show xml and sis updated by was me so at this point i am not updating i i'm not impersonating okay just wanted to see am i still the prince the sample principle whatever i called it school principle when we're done we can finish the rollback i'm not sure what's you want to try to step over or continue all right so i bet it would be the same thing yeah i it feels like it's the same technology here because it says paused at break point so it's like it put a break point after the snap before the rollback right i'm gonna guess it does the same thing too there's really nowhere to go there you go no more steps no more steps and if we go back and look at our student activity again 10 31 is gone right which makes sense it's done the rollback on what it needs to yeah rolling back a number would be dangerous because other people might be working on the table too [Laughter] i wonder who got to figure that out what works and what doesn't what's dangerous do we have any questions from the chat um not so much questions but more helpful information uh lisa is pointing out that if you want some good best practices for creating atf's or get started with headless atf testing there are academy sessions um that you can look up on that have a playlist on that information stuff like adding a timer to allow for flows to run approvals to be created creating test data at runtime when to use server step overs over ui steps etc um that's uh lisa's pointing out that those are all stuff that you can learn if you look it up well where were those videos when i was trying to do this last week did you ask lisa she would have always be right i learned this the hard way and then you tell me there was already enablement material on that so we're now telling you good stuff thank you lisa yeah those academy videos can be found on the community playlist so if you go to youtube um search for servicenow community yeah you'll find that in the playlist for the academy series good stuff oh she sent me the list [Laughter] wow that's just that against all the injury [Laughter] the rest of the chat is talking about a feature that came out in rome called headless atf which is basically being able to run atf without having to go in and pressing the button itself to say run test suite or anything and have it run on its own um that you know headless headless is becoming a more and more used tournament service now yeah we've got headless mid servers now too so using the docker technology you can deploy these things run your atf's uh scheduled without having a server farm full of different machines not a lot of fun stuff in there excellent i think that's uh pretty much everything we wanted to cover any any announcements you'd like to share earl uh other then check make sure to keep checking in on san diego it's running pretty solid here different every blog entry every creator toolbox every live coding happy hour every uh breakpoint podcast had to speak the word breakpoint in one more time on this last week's break point was with charnlin on some of the new developer tool updates in san diego that on the 23rd of march we'll be releasing another one with julia perlis talking about decision builder and decision tables that's a fun one and then looking at doing one on rpa hub a couple weeks after that i forget what that date would be early april 4th 8th something like that whatever the wednesday is on that week so we've got a few more to go with san diego a couple more weeks on creator toolbox and live coding happy hour then we're looking at bringing you some new video content right now earl and brad and i are heavily focused on the san diego content but also got to spend some time on the creator con content as well we've got some real fun stuff in the works too early to really socialize and share with that but that'll be coming soon keep your eye out for announcements we've got an upcoming developer newsletter so if you're part of the developer community you've got a pdi for example you've signed into developer.servicenow.com you'll get the newsletter we get some great articles and comments and what not coming out about that knowledge information is out in the open now so if you have been living under a rock and haven't looked around yet you can start looking up what's going on with knowledge seeing the changes this year knowledge.servicenow.com is where you're going to find everything and make sure that you uh check out the creatorcon page as well it'll keep you up to date there's even some quotes on there if you notice they're little dynamic tiles there's a library of about eight or ten quotes i've got one see if you can spot my quote that's it for today thank you very much earl for for doing the driving somewhere i have my sound effect but it's not at my fingertips i failed on that one so we'll put the outro music in post until next time everybody stay safe we'll see you again on friday for uh the facebook open id connect stuff and some tips and tricks we learned about that as well so great for citizens or or government organizations who want to say perhaps have a portal where citizens come in through their facebook authentication we'll show you how to do that and why essa the open id connect has been out for over a year but we didn't have facebook talk to you then take care everybody stay safe [Music]