Enforcing Coding Guidelines with Instance Scan
[Music] thank you [Music] foreign [Music] ER toolbox it feels like I haven't heard that music in forever it is so great to be here with you today this is the show that brings you our servicenow developers and admins all the tools you need to be effective on the servicenow platform and joining me today we'll do a quick introduction for those that don't know me I shouldn't always assume that there are people that are watching this is their very first servicenow video and thank you very much my name is Chuck tomasi I'm a senior developer advocate here at servicenow been with the company since 2010 I was a customer for a couple of years prior to that and have a developer experience that now extends past 40 years I have a hard time coming to that realization it means I'm getting old also joining me today is this man over here Sasha give yourself an intro hi everyone I'm Sasha and Sasha Vancouver I'm technical consultant with servicenow now for the best part of three years so I joined service now uh pretty close to the beginning of the pandemic guys still remember back that time yeah so the place started you know working from home and all these things um didn't have much of exposure to a service now um other than through the application process and then since then really grew in love with that wonderful platform now especially around all these topics around Pro coding automated testing automated code check cicd pipeline automation so these are the topics that really keep me keep me up and running and enthusiastic so thanks for having me thank you very much Sasha we have one more person joining us he just jumped out though I was going to introduce a great Fanfare waiting for me to come right back we sent it to me on teams at the moment so we're getting real-time updates as we speak this is a bit of a unique start for us huh it just froze he's having some technical issues so we'll bring him in in the meantime Sasha why don't you give us an introduction of what we'll be talking about here the title was something about coding standards guidelines and instance scan oh yeah yeah sounds pretty boring doesn't it um now you know um a coding guideline um you know most people who ever work in an organization and a larger team um everyone feels like you know bad about everyone doing its own stuff and doing things a little bit different and there's no consistency in the code and everything is difficult to find difficult to change and then some you know smart people come around and say hey here's a coding guideline you know you you guys all have to apply you know what what's written in here yeah and if not yeah it's it's going to have some kind of consequences well the fact is most of the times this coding guidelines sits in some kind of a Wiki or Confluence or whatever and it sits there well unchanged for ages and no one really looks at it right you know this is this is usually what we think about um a coding guideline now um something very important happened right Chuck I mean we waited for that so desperately long and finally we have it now in Tokyo release we finally have modern JavaScript finally introduced into the platform um and that's great we've we've all been waiting for that um but what does that mean actually uh it means that we have a number of new options a number of new tools and um I guess developer teams will adopt it very quickly yeah once it's available once it's in your development uh environment and and you heard something about yeah they updated production already so you're going to start using it so that means we will have more variety on how things can be put in code right so we have more different flavors more different tastes more people making things even more different not arguing about whether that's good or bad but but the variety of things that you will see will increase so if if any organization if you have ever thought about well if when is the best point in time to introduce a coding guideline that people really will respect I guess now is the best time to do that so yeah this this gave the headline um for uh the show today is how can we make use of platform capabilities to build a coding guideline and actually to enforce it right I mean taking that coding guideline out of this uh fairy tale sleepy uh castle Yeah in the Confluence where it sits into reality right so this is what we're going to talk about okay yeah and you're absolutely right this is a great time to reconsider those coding standard guidelines because as you mentioned there's more options with Tokyo we've introduced es6 plus you get more of the ecmascript capabilities from 6 7 12 2021 what you know pick your version and uh we will continue to extend that as we go forward with new releases but like you said that means some people are going to choose like me the option the way they've been doing it for the last seven years and other people will say well no we should really use this newer cleaner you know modern way of doing this say you've got to decide which way you do and you're here to show us how we can enforce that yes absolutely don't we already have like a quick start or something in in instance scan that we can use well let let me put it this way um the platform has instant skin yeah an instant scan can be used to um to build out your own checks those who are familiar with the health scan um I would say instant scan is the little brother of of healthcare well let's let's give them a definition of what what is incident scan for the people that are just watching for the first time right yeah let's let's take a look you wanted me all right screen share coming up yeah where'd my magic pot and go to do that so what is it actually um instant scan wait I gotta stop you you I think everybody needs to take a drink because I'm going to ask you to make your font size larger oh oh yeah yeah there we go much better all right so okay instant scan instant scan has something that is called a check and as you can see there's quite a number of checks um already in here uh some of them have been created by myself in in this instance and others are shipped with the instance itself now well this this doesn't really you know give us a good idea on what's really there so um there's something like comparable to uh the automated test framework where tests are organized in Suites okay we have the same thing in instant scan as well so um checks are organized in in Suites yeah like files are organized in folders sure and currently instant scan comes with um the one and only instant security Center definitions scan Suite with a number of checks that all circle around the security configuration of your instance and um there's there's a nice dashboard a workspace that that shows you the results of these checks and helps you to configure the configuration the security configuration of your of your platform better now the interesting thing for me was we can also use instant scan to look into our code well essentially we can use instant scan to look into basically everything in the platform if it's a record we can scan it um so what what I um thought what we could do is we could take all that Knowledge from the community my own ideas things that I see going wrong at clients I'm working for and put that into a a scan Suite that explicitly looks at code quality and this is where um yeah the code sanity Suite is coming from so okay in a nutshell instant scan is a platform capability it comes with a number of checks but they are security configuration related so anything else you want to look at uh like code or maybe even even data consistency all sorts of things you have to create your own checks in order to get there thank you so that's that's essentially what I did so I'm interested in code quality so I created a suite with a number of checks that look into uh code quality um I put all these checks and and the suite and the number of functionalities around it into a scoped app I called it called sanity it's uh out in the public it's on it's on the GitHub repo and maybe we can share the link um uh pretty soon and um uh yeah that that's that's uh basically my my starting point okay now I have to imagine that if this is your starting point but if I'm working at you know my organization uh we may have different rules this is some of these are religious if you know philosophical it's a philosophical that you know well you know Sasha said this but what we've got a strong stance that you know he says don't do this but we're really going to do that is there's like just an active button we can deactivate on the Sasha rules we don't like or create new ones that we do like how do how do we extend what you've given us great starting point by the way yeah and um one of these new um uh arguably uh uh religious uh code sanity checks that that I introduced um I personally for example yeah let's let's put a specific example here I don't like the spread operator that's it's one of these new modern JavaScript features um and whenever I see a code that is using the spread operator I I feel like okay it really gives me a hard time to understand this and when I ask around um people also have difficulties to to understand that right so my my conclusion my personal conclusion really was using the spread operator uh no matter whether it's used to to decompose arrays or to express that there's multiple parameters and a function there's even two flavors in it it does not really help me to understand the code better right okay but this is my personal impression that's my personal opinion I think a coding guideline should start with avoid the spread operator right okay opinion now the most straightforward thing you could do if you say well let's let's use that as a starting point install the app and run the code sanity Suite against our apps but um we totally disagree about um the the idea of not using the spread operator we love the spread operator it may be part of your base training who knows yeah well it shouldn't be illegal just because Sasha says so no in your world this is a bad thing because it's difficult to understand it would be like somebody trying to yeah you I I'll say it a thousand times but if you have an option of writing a coded version of something and a non-coded version a low code no code version of that same solution take the low code no code one because code always implies there's going to be technical debt if you do have to script and we do sometimes make sure it's maintainable it's readable this may be one of those situations that you know it it's confusing to people who don't have that depth of experience so you may want to turn this on and go hey you know this is important to us that you don't make it more confusing than it has to be if you wanted confusing code you'd be writing Pearl I'm sorry I had to take that job but yeah it's it's pointing out exactly the right thing um it's all about experience and even well I mean Chuck you you probably can uh you can claim that stance for me I'm not sure um some of us will consider themselves as experienced developers right right others might not and um in in every team in every organization there's you know the the hard-boiled seen it all roughneck kind of super coders who've seen it all you know and I have friends like that you know long gray ponytail long Gray beard they look like Gandalf yeah exactly these guys they don't have any issues with a spread operator right they they read it they immediately understand everything on the spot yeah no no worries but what about the the less experienced leads right um we want to you know get a new um generation of coders on board well but let's admit that sometimes it's the the younger coders who have even more experience and are better in writing JavaScript code than we are right um and maybe it's just me getting old right it's uh I'm not used to the spread operator that's new to me um yeah I don't really want to learn something new so um yeah it might be opinionated long story shorts yes you can deactivate it okay right right um it the truth is it's not as simple as that there's even a more sophisticated mechanism built into code sanity that allows you to to Define blacklists and whitelists for specific applications this is something that you uh May stumble into if you if you're applying it to a huge organization with an existing huge complex code base where you have to consider a certain Legacy that is not compliant with the coding guideline and will never be but you still want to see a clean slate um on on your findings right and then you can yeah and I look at that like somebody who built a house say 150 years ago you know they were lucky to have electricity if at all uh you know but as building codes have changed much like our coding standards you're looking for new things but you can't always go to you know a house built 150 years ago and say uh excuse me you don't have a ground wire on your wiring you can't live in this house anymore you know that's the way the house was built that's the way all the appliances work as they make improvements they can go back and say we're renovating this bathroom let's fix the wiring let's take the lead pipes out of the bathroom you know all the stuff that makes that older house unsafe and difficult to maintain they can fix along the way so it's it's sort of a maintained as you go but this will allow you to enforce the newest standards on the newest work exactly exactly and as as I said some of these checks were probably the majority is highly opinionated well yeah there are some of which we probably can agree quite quickly yeah but again that's assumption we should not have debug output in our code we shouldn't have sys IDs and our Scripts um okay check on that but here again throwing exceptions that that's that's a whole argument in it on its own yeah should we use exception handling or not in our own code uh bad enough that we have to catch exceptions from someone else but should we throw them in our code and you can imagine how you know how religious this debate can be but this thing has I built it for for two purposes the one thing is really to have a baseline on um on creating a code coding guideline and that it's not just a document that can that that actually can be used to to look into code and and give you the deviations that's one thing the second thing is I wanted to wake up a developer community at a client and really get them you know being part of the conversation and for them even these highly opinionated Highly Questionable checks it's it's a gold mine you really get the people in engaged and involved right they right they in love for example there's there's this one guy um who loves exception handling um he thinks that every error in every context should be thrown as an exception um he was engaged you can guess that right so it's very easy you know to bring people um to the table if you if you make some bold statements yeah let's get consensus on that like you said this is a living breathing set of Records it's not a static document that lives somewhere because like you said technology changes practices changes the experience level of your team's change you will need to come back and revisit this and say hey do we still need this standard this implies there's got to be some sort of review process in place that you want to visit annually quarterly that's up to you but someone should be in charge of these coding standards to take a look at these and maintain them over time because like I said there's there's change will happen and and you may say you know what today's the day we're going to adopt that spread operator because we've been trained on it everybody loves it we think it's the right way to go if we don't see it we can you know maybe we can flag that too but uh it's no longer an evil thing so we'll disable it uh that's that's the discussion you want to have so use this set as a starting point and then come up with your own you may have 15 20 150 more that you'd like to add to this and it allows you to keep that up to date far far better than this document that you assume everybody's supposed to read and memorize exactly and while while talking about a document um what yeah why not having it as a document wait a minute what kind of voodoo are you doing here did you write this or is this generated from those records it's it's actually well no I hadn't written all every single word but in fact I did but what we see here is um is a UI page that uh takes the contents from all these checks that are part of that suite and um and it pulls it together in a document and the fun part is that you can even you know just copy paste the text and paste it in your Confluence if if that's what makes you happy yeah um but at least you know someone who says well I I can see you're not all introducing all these checks and you know no one knows about these checks is this documented somewhere and you could say yes it is there it is oh all right okay so um this this leaves you with the the only burden that you have to copy paste this um this UI page into into your conference page uh once in a while or you even create um an integration to do that right I was gonna say I if there's a rest API I'm all over that absolutely absolutely right I mean yeah but you know you might run into issues because this is probably living in a deaf environment and your Confluence is well you know it's part of a production stack so yeah it might be difficult uh once in a while but at least it's it's a point to start as I look at this list I'm seeing some things that we've been pontificating or pre we talked about religious things before we've been uh we've been preaching about these like no hard-coded CIS IDs in scripts that's been sort of a rule since time immemorial documented since at least 2012 and to have it in here you know some some old familiar ones in here but yet you've added a number of them that I didn't think of 10 years ago so this is great this is great to be able to keep this up to date and again now this is a starting point that organizations can use to create their own variants on that I would love to see a community built around this he said you've got a GitHub repo let's do some full requests and and get a collective of hey I also found this or uh you know we made a new check for that I think that'd be incredible it's it's public on GitHub for a reason yeah it's it's public domain so they just search for like servicenow code sanity or what are we looking for um yeah um pretty much oh this is here we go that was tough [Laughter] simple service now called sanity okay how many Works do we have right now who's who's taking copies foreign I think not that many yet uh let's see uh where can you see the forks I thought it was in the upper right and I see stars no Forks upper upper right yeah right yeah it's relatively new no sure let's see what happens after the show all right um do you want to run this I mean how long does it take to run your Suite would it be something it depends on on how much is in your app okay now let me let me um yeah let's let's take a look at that in action but um first of all we need something to check and I I brought some another fun project of mine you you can guess what it is ah the good old game of life I had to write this one um it's it's really a POC right I haven't you know made it public on on my on my GitHub repository yet um now what it does obviously it plays The Game of Life and it starts with a pattern and um the pattern is basically you know one of these figures here combination of uh set or unset uh fields and and these patterns live in a um a pattern record that is derived from susmita data so I can make it part of an app and then ship it um but the first thing it does before it actually starts it needs to load the pattern right um so anyway um let's let's take a look at it so I'm I'm here in the in in studio and The Game of Life Application and there's an additional button um which is new yeah usually it's not there so it comes with the code sanity app and what it does is it scans the complete app against the code sanity Suite so you don't have to select it each time yeah so if you use the out of the box instance scan you have the choice between I can just show it yeah so if you are in a scan Suite yeah you have this execute sweet scan button and then you can select um for example a scope right and this would essentially do the same thing so you've simplified the process by enabling it within studio nice exactly yeah I'm still working on on getting a better progress dialogue which already shows you the um the button to navigate to the results um still work in progress but it does what it should you're way better at doing those process dialogues than I ever was I could never get those to work well I stole this one from I think it is committing changes in in studio okay so I kind of you know side by side compressor what are they doing how am I supposed to do it and yeah that's how far I got um so let's see so let's view the results so leads me to another page that shows a scan result record right I'm using uh um I'm I'm using the snutils plugin here over here so I can see the technical names yeah for those of you probably have spotted that already and the scan result record gives me an overview on the results as the name suggests of the scan it I do have 12 findings what the hell I'm not able to follow my own coding guideline or whatever we call that demo data yeah maybe yeah no one to be honest I prepared so it shows me which Suite it used it shows me the checks that it applied it reminds us that you have to use this photo there's a check um service portal widgets there's no service portal widget in that app so it wouldn't even use that check in the scan because there's no widgets to check sure um if one of the scripts within the checks would fail and run into some kind of an error you would see it here in the failures and here you can see the targets which in that case is my custom application game of life so out of these 27 checks I've got 12 findings and now I can do the work of looking at it and see if I can uh fix them um so I mentioned that here my Game of Life Application um the first thing that it does is that it loads the pattern so it um somewhere in there you have a HX request that goes through certain descriptions but the main function where it ends up at is the get pattern script includes right some kind of a pattern ID and um yeah this is where most of these findings are so let's let's take a look again so what findings do we have Glide record without new that's dangerous what the hell I mean this doesn't even look like a coding guideline thing this is more preventing me from making a mistake so um where is this oh look at that damn it I forgot the new and and this is this is actually a tricky one I was really um asking myself does it make sense to add this to a coding guideline because um it's it's I mean shall we cover every single mistake that a developer could do well probably not but this is so commonplace if you search your place for for example equal signs any number of blanks and glide records you will be surprised what you can find especially in cases like this see what I'm doing here I'm doing a request on my on my patterns yeah this need little uh Game of Life creatures and I'm using the ID and match it to the pattern and if I find it I I return it um and if I don't find it I give it another try and see does the pattern match the name so it's kind of a fallback right if if this didn't work I try that right now if if my implementation is is mostly using the ID and not the name I might never see that bug to happen and I mean as we just saw if I just skip the new this is syntactically correct yeah I I can assign the the class object that is representing the light record into GR pattern so you could do something like new gr pattern and assign it to another variable um so this is really dangerous you should be here so what else do we have debug output well obviously I'm still working on it so what's the deal but yeah I'm using my own function here debug but it will also respond to something like gs.mebuck that will have the same effect um so of course I don't want to have these little debug output things in my production code so I better remove them here's one and here's one here's another one yeah that that should be all or you could use a property to enable and disable that yeah um that's true and guess what there's there's even a um a check that checks if the system property that defines the lock level of an app is defined what here again you could yeah well you could argue no we don't want that system property to be part of every single app um then of course you can deactivate the check and just ignore it but um at least please have that discussion right maybe it's it's a good idea to to have this um have the system property in place but yeah you're right I mean that's just another way of of achieving the same thing um but of course you might also argue whether does it make sense to have all the um code statements that produce the debug output in the code in the first place if you're no longer debugging anything if you're done debugging um you may just you know want to remove the debug output I like having the property because there may be a situation in production where something acts up that you can't reproduce whether it's based on the data or some other situation you can always turn that debugging on see it for a little while capture the log statements and then turn it off and then the instance can check can act as a reminder that at some point you should also remove them yeah so it's always you know a question on how you apply it okay now what else do we have um oh the constant array index entity pattern that's a nice thing um what we mean by that is this year constant that is used to to um to get the value from an array um the point is uh why is this problematic because if if let's say in this case um the a pattern let's assume that that's an array yeah it comes out of parse Json so it could be an array it could also be an object yeah it depends on on what we put into the uh parse Json function here so we don't really know um is this really an array and is is using the the syntax a pattern in you know brackets zero is it safe because we don't even know whether it has the first element right maybe it is maybe it's empty and if it's if it's empty in that case uh that statement would cause an evaluator error and stock the script on the spot so even with modern JavaScript that that we have in Tokyo that there's um there's no something like um a conditional uh referencing yeah where you can say um only if uh that that element exists or that that member exists address it and otherwise return null yeah we don't have that yet so so this kind of a syntax is really dangerous so we shouldn't be doing this um in that case we want to know if a pattern is is an area in the first place so I just remove it but um if I think that that um I I need to do some some error handling or some processing of an array I should always use either an iterator pattern or a for Loop um or anything that that prevents me from from putting a constant index into um uh into into that array syntax yeah because if the element doesn't exist it would just you know produce an error and and stop the script and and guess what using that check alone and with you know being scanning a a large client's code base for a while now it revealed so many hidden defects that only eventually show up uh difficult to track difficult to find difficult to reproduce and we could remove all these dangerous the dangerous places in the code absolutely amazing okay what else um one more one more yeah we still have energy uh yeah the throwing exceptions um okay well in in that case you could say well this is this is a bit um um constructed yeah and again to be honest maybe it's just my age or you know I grew up in in in early sea uh you know world where exception handling I don't know whether it still exists back then or already existed back then I don't know but anyway I was used to use return codes to um to communicate error conditions yeah I I returned what the function is supposed to return or a return false and lock an error somewhere but um and I'm always thinking about you know the v-shaped execution flow right I'm digging deeper into into more functions and then the functions are something it would return the return code and return orderly yeah I don't like dig deep into functions and then directly um throw the exception until it's been captured so again highly opinionated you can have a different opinion on that and probably a lot of you out there do have a different opinion on that using this example here to show that how um let's say how unnecessary um exceptions often are you know in this case I could just you know return false and and right um of course there are use cases in which uh you can justify using uh exceptions but in that case obviously it doesn't make sense so um let's save this and run the check again it is going to take a little moment while we're looking at this you might have wondered um how did that build this um here's here's a fun fact and if we have that two minutes I'll I'll uh sure lead you into the rabbit's hole it's it's a react application witchcraft I say I I let that sink in yes it's a react application um let's take a look it's uh is it this one this one so this is the code of what we're currently seeing here this is this is our pattern ID yeah so it initializes itself using a particular pattern and that is then loaded using an Ajax request um this is a shorthand for uh react dot component yeah so it's just has its own parent class but it's it's using the the core react feature here uh here it's referencing to another react component this one is doing let's say the actual game state it loads the pattern and it it draws The Game of Life board which actually does the drawing um nice we can see it here so obviously I'm using SVG as the render engine um and using uh let's say the usual uh react typical patterns to put the different HTML elements together so that's what the Game of Life Application does so in case anyone asks can we do react based front-ends in service now yes we can all right so it's finished hopefully we have less than 12 issues again or 12 findings I hope so as well let's see oh yeah we fixed some there's still some remaining right eight weddings so as you can guess now next iteration we will look at it again uh see what other findings are there and fix them step by step and now the the big question is I mean we we talked already about um it needs to be embedded into um a developer community a group of people who have that discussion who form an opinion about which checks which they have which should we not what additional checks do we need um eventually also trying to to learn from Big mistakes right see what did we do wrong last time when this big big release just you know was delayed or new defects were discovered can we derive something from from the mistakes we made can we create a check that prevents us from making comparable mistakes in the future so this is always a very important question to ask how can we build in the experience of this community into new checks because this will always be new team members there will always be junior developers helping out um so errors will mistakes will repeat themselves right if we if we find a way to to capture them in in checks that's awesome uh Sasha this is built on instance scan so I'm assuming you you wouldn't have any issues integrating this with Tokyo's app engine management center which has configurable pipelines and and one of the options for those configurable pipelines is hey when I'm moving from Dev to test for example I want to trigger my ATF test I want to trigger my instant scans this could be part of that natural progression of your configurable pipeline in app engine management center Etc or if you've written your own pipeline configuration same apis to trigger instant scan it's just you're triggering your version of code sanity so there's nothing magical here there's giving you a whole bunch of starting points to test this with absolutely um while talking about apis and pipelines um I think it's it's very uh important to know here that there's this namespane namespace snc ICD yeah it's a continuous integration continuous development and um we do have a number of apis here we do have one for ADF we have one for um installations and also we have it for for instance scan execution nice so yes no matter what kind of of tool system or I don't know integration mechanism you use um it's accessible via rest apis so there's no excuse yeah you can build it into your CI CD pipeline no matter what yeah and you know the the the next interesting question is how do we treat these findings from an organizational point of view and this is something that that we're currently developing um with one of our clients is that we want to get to a point where there is zero findings during an automated CI CD pipeline deployment right and and this means that the teams really need to do um a lot of homework now in order to get rid of their findings or raise their hand and say hey we do have a legacy code base we totally agree on the finding themselves but it's just too risky it's too complicated it's too uh too much effort essentially to fix it so yeah we can we can create an exception or you know dump the check uh completely because you know the balance between um how much gain do we expect from that check versus the the effort to be compliant is is just too high um but we want to get to that point where where we run these checks and we get no findings at all wonderful stuff Sasha you've covered a lot I didn't see a whole lot of questions in the chat for us other than hooray thank you thank you once again there's been a lot to look for yet if you want to find this I'll do my best to put a link right to his git repo in the description of this video shortly after we finish recording this so you should find it there if you're interested take a look at his code sanity implementation and give it a try test it out on your PDI write some code see what the findings are discuss it with your team there's your call to action for this time so thank you again and until next time I look forward to have you thought about making any of this uh a session at the next Creator con oh yeah it's submitted I'm still holding my thumbs okay who knows maybe we'll have some Hands-On for those that that say I want to test this with my my own hands in person you've got an option you can either do it on your own on your PDI today or if you're heading to knowledge in May of 2023 don't know when you're watching this but we'll see if we can have something for you there thank you very much and uh where's my magic music where we wave smile and Say Goodbye thanks again bye everybody [Music]
https://www.youtube.com/watch?v=05z4XamuC3Y