[Music] hello and welcome to creator toolbox on this uh tuesday i almost april 24 20 april 20th that's important of course uh so uh let's go ahead and get started with introductions andrew hello everybody i'm andrew barnes developer advocate here at servicenow uh i have been developing on the platform for about seven years and one of my specialties is really relevant to today uh which is managing uh development and deployments uh for servicenow so uh that is uh going to come in handy today all right my name is chuck tomasi senior developer advocate at service now been here for well since 2010 building all kinds of applications and videos and blogs and bringing them to you to share the excitement to share the education customer for a couple years before that and have oh it's almost 40 years of experience in this industry all right thanks chuck my name is brad tilton i am a developer advocate with the developer program here at servicenow i've been developing on the platform uh for a little over a decade and excited about today's show before we get into it we have any announcements we want to make i do have one quick one uh the americas certified master architect program is taking applicants until was it the 27th i think it's just another week after we broadcast this go to devlink.sn cma if you like we can put the link in the youtube chat as well but devlink.sn cma it is a paid for certification it is quite the time commitment but you can read all about it on that page if you'd like to engage with someone who's been through it before either in ams or apj we can connect you with somebody on that you can ask more about that so very elite very difficult i'm not going to couch this it is it is a challenging program not everyone who applies passes so uh there you go the other thing is knowledge is coming up uh next month uh so head over to knowledge.servicenow.com and as of today um so prior to today you could register and you could favorite things but you couldn't really add them to your list um so today you can go uh add them to your list so even if you previously favorited them you'll want to go and and interact with those records again and add them to your list so that will be for the on-demand submissions are available right now and i believe all of the scheduled ones are available or soon to be available there as well all right and then we also have a hackathon coming up creatorcon hackathon that's going to be happening on may 8th i know that absolutely and so for the hackathon you'll go to the same place knowledge.servicenow.com search for hackathon and find the hackathon session and join that and then you will get some notifications about uh how to create your team uh shortly thereafter uh so it should be uh it will either be immediate if you're watching this in the future or it might be a day uh if you're watching this live today all right well without further ado andrew why don't you tell us what we're going to be doing today yeah so we're going to do a two-parter today so this is part one of managing your development pipelines and part two will be next week so today we're going to talk about the slightly older method which is update sets and i'll get into more of that in just a second and then next week we're going to talk about source control and the application repo and so what i'd like to do with this is talk about the maturity model of how to go from wherever you're at today to to a more mature development pipeline setup so we're going to look at update sets today because that's what primarily most of you are still using and there's definitely maturity that can be done in the update set space for you and then next week we're going to look at the application repo and source control um and how to move past update sets and uh start using those to get even more automation um and uh you know consistency and deliver fast delivery in your development deployment because i think developers should spend time developing and not time managing the release of their development work no value in all that so wherever i have worked i have worked on updating uh our development pipeline to minimize the amount of time that developers have to spend on that because i prefer them working on stories then managing getting their stories into production uh and so do you either of you have any questions about that before we start the screen share now just a quick update for if everybody's just watching or new to the platform an update set really just records those changes you're making to the configurable objects in the system so a business rule an acl pieces of your application or an existing application that you want to configure modify create perhaps you want to make a new business rule or a new role you can add those those are recorded almost think of it as this stenographer off to the side going oh yeah we got a new business rule it's collected into this bundle called an update set that you can then move to your other instances or share with your friends if that's what you do and it it doesn't record what i'll call transactional data a user that's added to the system an incident that's created so certain tables are watched for those recordable changes and we're going to show you how you can manage those and move those from one instance to another this has been around since 2005 2006 i didn't get the exact date when i talked to david lu the other day but they are a very old-school feature they're not going away in light of the newer methods that we'll cover on the next episode but just for a little background if anybody hasn't really been made aware of what an update set is awesome thanks chuck anything from you brad no i'm ready right into it i will start my screen share then all right so uh the first place i thought i'd land us and we'll come back to these is uh i've written several blogs on update sets and managing development pipelines so over at developer.servicenow.com you can find the blog and if you search update set uh you will get the tag that will get you to these two uh blogs that we'll look at today so there's update set development management and then there's managing update sets um we're going to look at both of these at different points today so uh with that let us uh and and here's how i actually pulled them up myself i went to developer site i did a search for update set and that's how i found them so that it that is how i i will find them as well as you can so as a developer it doesn't matter you know which application you're using to manage uh your uh stories or or even if you're calling them stories that doesn't matter i'm going to use the servicenow story table today uh but none of this is specific to the servicenow story table it's just the examples of buckets of work that i'm going to do so i'm going to use servicenow tables but if you're using any other product you know it doesn't even have to be an application it could be an excel file it doesn't matter the the principles are still going to hold you've got some you know requests for development and they're going to be grouped together in uh larger bundles and then you have releases uh which is the actual deployment and so we'll walk through that um so let us open one of these stories and these are all just fake stub stories uh which is for real these aren't real i'm not gonna build a laser today i love lasers i've built lasers but we're not gonna create any lasers in servicenow today um so when i have a story in servicenow i do a one-to-one relationship at minimum between my story and an update set so right off the bat i'm going to go with a naming convention so i am going to go create an update set up for this story and so you're doing one per story yeah i do at least one per story okay and we'll talk about why in a second so i've got my naming convention which is uh my story number and then short description uh it you know you should have some form of naming convention for your store for your update sets this is just an example one so follow whatever your org has but there should be one and it should be in some form relatable back to the the story record that you're working on andrew can you enlarge the font a bit absolutely bueno so when i submit make current that now sets my update set that i'm working in to be that update set and that is what i want so i'm going to refresh this window so that ui 16 tells me the right stuff at the top so now i'm going to go uh create a business rule so right this moment in that story record we will see this customer update table so this related list is that stenographer that chuck was talking about so down here is where we're going to see the entries for the work product that i do so if i go create a business rule and i'm going to do this directly in the platform view i could do this in studio just as easily they'll still get stored in my update sets so it doesn't matter how you're generating these records um they will still get stored in that update set table um in in that related list so we're going to create the laser party and we're going to create this on problem because lasers are a problem the problem table and well let's make it advanced because we like advanced uh and insert and set a couple of conditions like the state is assess and the short description the description contains lasers all right so when i hit submit that's going to create this business rule application record and application records are in business rules specifically are one of the watched tables that chuck was talking about earlier when he was talking about what is stored in update sets so my expectation now is if i refresh this related list i will get an entry here and i do and it tells me that it's a business rule for and and gives me the details of this and if you open this up you'll actually see inside of here this payload field contains that whole record that you just generated so all of this payload is the same thing is on this business rule that i just created if i exported this as an xml did i not click it come on you can do it if i exported this as xml or just viewed the xml which is similar this is the same thing that's stored in that payload so all of the details of that application file are here and it has to be because this is what actually moves between your servicenow instances so i think that's enough about the bare basics of update sets so let's talk about managing them a bit so in my update sets as i said i like my stories to have at least one updates that related to them and i've even you know done some nice automation with that but for today we're going to talk about the process and processes can be automated but they can also be manual so we're going to go through them manually but just note that you could automate some or or all of these things today so what did i mean by just you know maybe one uh story or there could be multiple what happens then um so for that i'm gonna return to my story table and i'm gonna pull up cake i like your apps and cake has got some scrum tasks and they're going to be assigned to different developers and so they're going to be working in you know in their development in separate update sets so one of the developers is going to do the shopping and one of them is going to do the mixing so update so local update sets uh let's get the story number and we're gonna make a new update set for our new story about cake we're going to save this and then we're going to make a new one for so dev so i've got one to my story and and uh chuck is gonna go do the shopping so chuck is gonna do our shopping for this and we might even take this is again just an example but we might take the uh task and that might be our naming convention if we have scrum tasks uh it's probably a little redundant i'd probably just use the scrum task numbering but for for this purpose uh we're going to be happy with that and then i'm going to create one more for brad who's going to do our mixing i got both my shots i can go out now and go shopping so brad is gonna do our mixing and now we have three update sets for this story and do we want to this is going to be a rhetorical question but it's fine do we want to actually you know have to uh you know export all three of those update sets and interact with them separately uh when we're in our release process and oh heck no because you might get them in the wrong order yeah there's there's all kinds of room for human error there the one thing you do need to pay attention to is when i'm doing work on that one update set i have to go and select that as my update set i got to make sure that i am using it it's recording to the right page in the notebook so to speak absolutely so um i've narrowed down my update set uh table to just the ones that start with my story so you see how this pattern is helping me with my problem right here already these are all interrelated and we have this thing called batching and batching is a way to group relate update sets together and then be able to act on them together and the way it works is very simple it's the parent field so if you're familiar with the parent field uh from other you know record interactions in servicenow update sets have the parent field as well and they create a hierarchy and so our story one is going to be our parent for the other two so we're gonna set that we're gonna set that there we go so what that's now done is made the parent uh this this story is now a batch so if i open up this record it'll actually give me a little bit of different interface down here at the bottom you can now see update sets in this batch so it now knows that it's a batch and shows these related lists of the updates that's in the batch and you'll note there's three here because it actually counts itself is inside the batch which it is it is part of the batch and the children are here and uh one of you could potentially ask me this question but i'm going to answer it anyway can i batch batches and the answer is yes and we're going to show that so you can have a whole hierarchy of uh these batches and i've got some images for that here look at that so our story you know is that is that batch that we just created and it has those those scrum task update sets in it and our release is another update set that all of the story batches point to and you can also insert another layer in there for epics so if i return back to my instance and look at my stories and remove this filter i have to go to the story table to look at my stories you'll see that i have an epic so my epic will have stories in it and each of these stories will have an update set or or there'll be a batch so they'll have at least one update set and then my epic this one this laser powers will also have an update set and it will be the parent of all of those story ones and then our release again will be one update set that's a batch so all of the epics and stories update sets that don't currently have a parent will be set to the release update set as its parent and with that i can craft that single batch for release so let's walk through what that looks like uh in practice so we've got uh three update sets right now right or did we make four i think we made are they all still in progress or did we complete them they're all still in progress okay um so we're going to uh pretend like there's stuff in all of them so let's return to our update set table really i went shopping really you did i believe you you gave me money so uh let us sort by created all right so we've got our four update sets um so one of them is is for a story that's got multi you know so this is a batch this one's just one update set um let's us create a release update set so i'm going to open up that release record and get its number so you see how that pattern helps me uh having that established pattern is going to make life easy for me soon so this is going to be for the lab code happy hour uh release which is a little redundant name calling a release a release uh but it's fine so i'm going to submit that and now when we refresh our update sets i did i update i updated it i gotta make a new update set now you didn't catch that somebody should have caught me being bad sorry uh that's fine uh so uh which one does that first one the first one let's put it back so new one for lasers submit and now we've returned to our update set table um and let us filter yeah there we go give me a give me a nice filter so these are all of the update sets that i've created for this release and i'm going to apply the same logic that i did for the scrum tasks for the story to the release which is i'm going to look at my update set table i'm going to find all of the ones for my release and there are some tricks uh to doing this well you can get to the part where you actually like relate the records and then you could do a query that way for today i might do something like open up my release and see all the stories in there and get the story numbers and then use that as a query for my update set table since i used a naming convention that is something that could help me right so all of the stories in my release i could do a query for the update sets that have those those story numbers and narrow the list down to this so we're going to pretend like we just did that so we've narrowed it down to all of the ones in our release and now we're going to look at all of the ones that don't have a parent so these are all of the top level ones whether they're a batch or not and i'm going to set the parent of those to my release so now i have a batch for my release typically where you're going to build this release is in not your dev instance it's going to be in your test instance so i didn't do that this time but i totally could have and in fact i'm going to because let's do that so the practice would be to move move your individual stories to test and test them and then when you're happy with what stories are actually going to be released then you batch those into a release and promote that to prod yes yes you did so i'm going to as a developer when my story is ready for testing um in in my typical setup that means it's time for me to get it peer reviewed and for me the way that i've designed it previously is during the peer review process the other developer will move it to the test instance so they peer review it when they you know so they look at they inspect my update sets they check to see that things look right and then they promote that update set or batch whether you know so if it's a batch they'll promote the batch if it's a single update set they'll promote that and then deploy that to the test instance and then we'll go through our release management testing process on that test instance for everything that should be going into our next release and the ones that that do get approved to go into our release we batch those together and then we'll move them to our next instance so let's do that so we've got uh these uh three things that are dev complete and now i will go to my test instance and retrieve them so i'll go to retrieved update sets i'll go to update sources [Laughter] and i'll say retrieve completed update sets so just to back up a little bit the update source was defined once on each instance to say where am i getting this from do you recommend prod gets from test or prod prod source is dev so prod source should be wherever you're building you know so one step down from prod in your development platform it should not be death so it might be you might have dev test qa prod or you might have dev test prod um and so you know prods should retrieve from whichever one is right before it in the pipeline so i've retrieved my completed update sets uh so you'll see slightly different wording for the batch one right so this is previewed in a batch so actually i only have to open these two because these other ones are part of the cake batch so i will commit lasers the preview by the way when you do an import from an update source the preview is automatic yes it is it's not automatic if you do a retrieved update set like you import an xml file that your friend gave you or you downloaded from share put that in you still need to preview it and recognize if there are any errors you need to resolve them so one other thing about batching that's great and i didn't show it here but i could have um is you can have multiple scopes in the same uh batch that's not a problem so my top level one could be global and then one of them could be uh you know security incident response and the other one could be hr it shouldn't be those those three probably should not be in the same story but uh they can be in the same batch so we have a question uh what if you what if you're not using the update source and you're moving them locally can we can we move bashed update sets by exporting to xml absolutely you can so um when you view that record in your development instance so we will return to our development instance and open up the batch which is the cake one you'll see the up the ui action is slightly different for a batch which it'll say export updates at batch to xml so that will bundle up all of the things in this batch into a single x export xml record nice not sure i'd recommend that on a day-to-day basis for operations oh no i i mean it's much easier to just use the retrieve update set uh from the update sources um action um but uh it is definitely possible to move batches so i committed that batch which deployed both this uh updates it and its children and so now one of my testers uh you know has done some testing and they've said oh story uh you know 1005 for cake has an issue what do i do promote it to prod [Laughter] that's how we keep our paycheck coming so what we do is we return back to our dev instance and we make a new update set so we're going to call this one a new update set and we're going to say this is fix you know for xyz and i'm going to submit and make current and i'm going to go to my business rule and i'm going to update my condition to say or contains cake so i've updated my business rule to fix the issue um and now i have an update set in dev right here fix for issue and so i've completed this and now i'll go through that peer review process again and so for me another developer would come in here and preview this and check to make sure that that i have done the appropriate thing in this business rule and we will complete this record and then we'll retrieve it in to our test instance so i'll return to that update sources table and i will open it up and i will say retrieve completed update sets and that could have just as easily been a batch and so we've got our previewed fix right here and i'm going to commit this update set because i have no collisions and here's where the magic comes in does does someone know out there in uh the the comments land what i should do now so i have fixed an issue for story 1005 in dev and moved it to test and it test also has some other updates that's related to story 1005. back to mall batum all is what we shall do chuck that is what we shall do in this scenario you only modified one update one business rule but that first update set may have 16 things and the next one was only one little patch for that so if you didn't include the one with the bug you'd be missing 15 other useful things correct that one fix is not inclusive of all of the work for story 1005 so it's it just had that one fix for that one business role and as you said chuck there could be tons of other stuff in these other update sets because it is not holistic for all changes it's just the changes you made in that one update set and so i'm in my test instance now and i need to put story 1005 uh fix for xyz into my batch for story 1005. so i will go to the local update set table in test and i will find that update set that i just committed and i will set its parent to be story 1005 cake and so i want to make it consistent with these these friends right so here is our batch uh base and these are the ones that came over initially and now this one is here and test as well so now i have a batch in test of four stories and now we'll do our release i we did it uh in dev but now we're going to do it in the correct spot which is here in our test instance so we're going to make our release batch there's going to be nothing in this batch right now in this update set because we haven't done anything yet and i'll do a similar property that i did before is i'll do a search on the local update sets in test that match that should be inside of my release and then i'm going to only show the ones that don't have a parent because if they have a parent field that means they're in a batch and they're already gonna be grouped up for me so i've narrowed it down to just the two top level objects that need to be put into my release and now in my test instance i have built my release and it's still in progress and that's okay because i'm still might have another story that another dev is working on so i believe i had candy uh that that brad's being a little slow on and he hasn't yet finished his story but he's gonna make it before our release deadline and when he does we will add his batch for the candy release or the candy story into our release in test and then when this is ready you know so when you know when we're actually like solidifying what's in the release and we're going to lock it down so that we can do our testing in our qa instance we'll set this to complete this is going to prompt us to say hey this will close any in progress update sets in our batch which is no problem because there aren't any open ones because we're in our test instance and we don't edit our update sets in our test instance so once i hit save now this batch this release batch is ready to be pulled into my qa instance and run any of my testing there and if those pass i can then pull that batch into my production instance and preview it prior to my change window now why would i preview that in production but prior to my change window any any takers i mean so you can you know so you can see if there's any errors in the preview before you have to commit it absolutely that is a star uh for brad today on that one is um is if i preview it before my change window then i can you know now it's probably i'm previewing it during regular business hours and i can get any developers that had any work related to that uh conflict that might have arisen to help me remediate that whereas if i wait until my change window either all of my dev team needs to be available during my change window which boo i don't want that or i'll have to you know work harder to figure out that remediation during my change window and that will raise my anxiety whereas if if instead during my change window what i have is this release batch just sitting in production previewed and ready and all i have to do is hit that commit button that makes my release process much much happier on uh my my tummy it won't make it upset raise my anxiety so uh with that that is the main uh thing i wanted to show today about how to batch and how to build that relationship between stories epics and releases to your update sets but i've got one bonus did somebody somebody want a bonus sure we like that's why we watch these videos it's for the bonus material let's do one bonus uh so it is coming up on clone day and i need to clone production down to my sub prods what do i do how does batching help me with this and the answer is um we will we will repeat a very similar process uh to what you just saw for building a release but for cloning so in let's return to our dev instance into our update set table into our update set table and we're going to look in our dev instance so this is my dev instance this is my clone target and any of my in progress work that's here i want to make sure i save that before the clone right does that make sense because i don't want to lose any of that work so what i'm going to do is i'm going to create a new update set in my dev instance that's going to say clone in progress and then give it a date like april 20. and so i'm going to submit that and i'm going to filter my update set list to be in progress ones that have no parents that are not the default set and i'm going to then batch all of these into that in progress update set for the clone and this is going to throw me some errors because those are different uh scores those those are different scopes so it's not not going to let me do that uh easily i'm going to have to switch my scopes to do this but we'll do the global ones to show you what i mean in fact we'll just do a couple we'll do this one to say clone and then we will do this one and say clone and then i'll go switch my to what was that book now yeah and now we'll be able to do book now so you can mix and match different scopes within a batch absolutely and once i have done this for all of these so i'll switch my scope and i'll set these for all of them now i will have a batch for my clone which contains all of the in-progress update sets and so i will complete this batch and i can either promote this to prod and not deploy it prior to the the backup for the clone or i can export to xml and then after my clone i will either it will already be there because i imported it to prod without deploying it or i will uh you know bring it in from a file and preview and commit it uh so let's let's actually do that let's walk through that process um so let us switch our scope to global and then this will be happy we'll say complete this yes i really did mean complete it i'm going to export it then i'm going to perform my clone and then i'm going to import it back so pretend this is my dev instance uh it's not but pretend this is my dev instance after the clone i'm going to import these back so i should have clone in progress april 20 and we should go to the global scope so this is previewed or not previewed this is exactly what chuck said if you do it with the retrieve method uh it will auto preview and if you bring it in from a file it will not so i had to preview this uh and then i have to remediate the issues okay uh so let's uh skip the remote update and then we will commit this update set batch and then the last step that i need to do is disassociate these from this batch and open them back up so in my local update sets that were just generated i can filter to batch base i can remove their parent i'm sorry you don't filter on batch base uh it works in this case uh what you what you need to do instead is parent so ignore what i said about batch base you want the parent field to be clone and then we're going to remove that and we're going to set these back to in progress now you don't have to do this in progress part that's up to you in your practice but what you want to do is make these your parents for any more work on on these uh either way so i i'm okay setting these to in progress it's about the only time that i'm okay switching a local update set from completed to in progress is part of this clone process so now my developer who's working on this build tracker record that was in progress right before the clone is now back in my dev instance in progress right after the clone and that uh with that uh i will uh stop the share and then see if we have any further questions once i can find how to stop the share there we are and now we're back do have a couple of questions it seems like there's maybe still a little bit of confusion around uh how batching handles you know updating the same record and different uh update sets in the batch compared to merging update sets where we know that when you merge them you just get the newest one yep so the way it works in batching is a similar process it will you know apply the newest one but the benefit over batching over merging in that case is you actually still have the hierarchy of the update sets and the individual entries and that allows you uh the ability to back out different parts of your release so if you think of your release is that tree of those update sets um you know so this is a batch that releases a single batch and then there's multiple update sets which themselves might be batches underneath it you can back out any one of those but if instead you did that merge process to create that new update set i you can only you get all or nothing if you want to back that out you have to back out the whole release which is why you don't tend to use back out if you're using merge because uh you'd have to reverse everything whereas with updates at batching you can back out just one leaf or one one branch or the whole thing that's handy that's handy andrew any insights on how fixed scripts are handled and run in this process as related to updates at deployment yeah are they run when it comes to batching does it go through the whole hierarchy and run each fixed script at the end does it run it as each leaf is completed so my understanding uh is a couple of releases ago they stopped fixed scripts from running automatically when deployed to the administrator you got to run those fixed scripts manually after you've made your deployment okay that's that's my understanding and i don't believe it matters how you're deploying whether it's from uh you know the application repo source control update sets or batches okay we can test it real fast but but i i've got a little note in my brain that source says fixed scripts don't work quite the same way and i believe that means they don't automatically run anymore any other questions you saw from the chat brad um there's a question is merging going to get deprecated uh i don't know batching has been around for quite a while now and merging is still there so i i don't really expect it to get deprecated but you never know we we don't have an expectation that it will be uh but batching uh to me is uh handles everything the mercy does and has other features so i don't do any merging anymore because there there's just not a good benefit to merging because you can treat a batch as a single update set uh roughly uh except for when it's better than a single update set you hit a good topic here andrew we got a lot of discussion going on there was there was a point about can you filter when you say retrieve completed update sets can you filter what's coming at that instance because it may not be yeah i i really want those six that were in the batch but i'm given 200. any thoughts on that so you you can't filter that list um it will go retrieve any of the ones that it hasn't retrieved before um so there is no filtering that's applied you can filter your list you know after it's retrieved the only other way to do filtering is in the source instance so the one that they're retrieving from um any of the ones that are not completed uh won't get retrieved and that could be in progress or ignore so ignore it is a good one yeah i like that ignored i have seen yep um so post clone from production to dev one thing that is beneficial to do is to set all of the um completed ones from production to ignore um either you can do that in production prior to cloning or uh if you don't want to do it that way you can do it in the dev instance poster clone i prefer to do it in production so i set all of my update sets in production to ignore after i have released them um because then after a clone the only things that will be retrieved from my test instance from my dev instance will be that new update sets that i've created yep there's also another property that i had a hard time remembering but mark put it in the chat that says don't retrieve updates that's completed if they're older than so many days so if you don't want things there's customers that have been doing this for you know 15 years now imagine how many updates they've got and you could say look i really don't want i'm not interested in anything older than 30 days or 90 days or 180 days totally up to you what you set that property to there's a property for that there is a property for that um i wasn't aware of that property probably because i set all of them to ignore and it wouldn't i don't i don't need to narrow it down it's because they're they're already ignored and so it doesn't it doesn't retrieve them because they're ignored already um that's cool i did not i was not unaware of that uh feature thanks mark and chuck for sharing that i didn't know either it's hilarious how you know your decisions can make it so that you don't even know a different way to solve it because you don't run into that because you you don't get a long list because you you've set it up in a way that you wouldn't get a long list anyway i think i picked that up on an internal toi a few years ago it's only been around for three years or so but like you it's one of those nuggets that you stick away in the back and go i don't use it today but [Laughter] awesome all right well well thank you so much yeah thanks for demoing today uh again you know dhruv had a comment in the chat that he wants to go away from update sets and uh next week is your week drew we're going to look at managing dev part 2 but this time with source control and the app repo so thanks a lot andrew for dinner perfect segue to end andrew thank you andrew it is thank you all right anything else before we leave no just the music all right all right well thanks everybody