uh welcome everybody uh my name's Richard hor I'm product marketing for devops capabilities at service now and uh we're going to take you through a session we're calling reduced configuration related outages with a new product or new component of itm pro called devops config so with that let me hand you over to Joe and he can introduce himself and get us started over to you Joe thank you thanks thanks Richard uh so Joe offenberg here I been with service now about two years I'm a principal outbound product manager in the devops Buu and uh I actually came over with the acquisition of a small company called swle uh back in July of 2020 and sgle had a devops uh config data management platform specifically uh for devops pipelines right so you know we we will cover this uh today but the uh that technology has now been rep platformed into service now and it's being released as part of the Tokyo release and uh it's available now for download in the service now store and you can install it so that's what we'll be uh covering today yeah and just before you get going in then Joe I just remind people or or let people know feel free to put in questions anytime in the Q&A panel and we'll get to those as we can throughout the call uh feel free to add comments to the chat too we'd love to hear your experiences as we go through this all right uh so before we go any further we have to uh start with the Safe Harbor notice of course uh you know we might be making some forward looking statements and some optimistic statements in some cases and uh we need to show you this slide done okay also uh we have um these you know a series of these webinars based on uh the capabilities of itm pro uh so you can you know uh scan the link there and learn more about the other webinars that are going on uh ALS Al to support uh these new capabilities few housekeeping notes uh please keep yourself on mute if you do have questions use the chat right uh we will have time for Q&A afterwards but but you know we're not a huge amount of attendees here so uh feel free to um you know ask questions in the chat and and Richard will be standing by he could help with some of the answers or maybe we'll jump in and answer those in real time uh this will be recorded and will be made available to the service now community and uh there's a short survey at the end so we really appreciate your feedback after the after the webinar is over okay so I'll start talking about just you know what is the scope of the solution what does config data mean what are we talking about here um you know as as we've transitioned to the world of devops right one of the first things service now did was create a solution for managing devops changes and and you know devops is about small incremental changes happening continuously so we had to uh augment the change management process to support that and that and that maybe you saw a previous webinar we covered that in uh devops change velocity that's the product there uh but a big part of these devops pipelines also include configuration data and what this means is you might be provisioning environments in the Pipeline and and those environments might be provisioned with tools like cloud formation at AWS or terraform or kubernetes and the configuration of of like application components uh can become very complex especially when you realize that the same component will actually be used over and over and over again with slightly different settings and with a lot more complexity right so what we've done is with with originally with the swle technology and then now that it's been repl form into service now is we created a way to to manage this complexity to organize the data in in a in a friendlier way and then to also provide some validation around that but first you know what is the scope of this what is the scope of this data right so it includes application data uh often in let's say microservices or other types of applications you have you know feature toggles connection strings and so forth that are specific to the application itself uh it may be in like an application properties file or config.xml file these are just some examples but that's an example of application config data uh from One release to the next you might have different types of dependencies on on other components so you're going to have release related config data often this might be in a in a maven file or pom.xml file some Java files uh that basically dictate what files Go hand inand with other files in the solution and this is extremely valuable uh to be able to track this because it it can change from one release to the next and and if you don't have an understanding of which releases of which applications go with which other releases of other applications uh you can create some some some problems and these are essentially config data problems right uh now as applications and and move through the environments you'll get uh environment changes that require different set so maybe in development we know that developers like debug turned on but we also know that in production debug creates a performance issue so we turn it off right so but that's an environment specific setting you want it on in development but off in debug of course you're going to connect to different databases in development right so the connection streams from the database to I mean from the application to the database is going to be different uh you're going to have uh maybe certificates will be turned off in development right where you don't have to invalidate certificates when you make connections between your microservices very common for developers to turn that off because it's difficult for developers to get valid certificates sometimes H so they turn off certificate validation so these are the kind of things that are environmentally specific and then when we look at really the the the total swath of data that we have most of the data is actually infrastructure data these are the sizing of of cloud resources this could be uh the the shared storage the network devices maybe network is code devices this could be um load balances and so on you know and and all of those uh Cloud resource will have their own configuration settings they're in yamamo files they're in Json files they're in any files property files all of that data makes up uh the the the the service the business service you're providing right especially if you're doing it in the cloud and especially if you're doing it as code now what we also recognize is that this is becoming um more and more of a problem it's becoming more and more complex uh we have some applications you know one applications could have 40,000 what we call config data items these are the actual key value pairs some of uh some of like the bigger applications like an e-commerce platform might have 250,000 different config data items so these are actual settings in that application and then of course like a financial trading system could have over a million configured data items that we have to keep track of and and what happens is uh with the proliferation of things like uh infrastructure is code and uh tools like terraform and kubernetes it becomes very easy to make these changes and redeploy the environment over and over and over again with slightly different settings so how do you know which settings work the best how do you know how far you're deviating from your Baseline uh you know all of that needs to be tracked so we see this is four different states of configuration data right so first you there's the definition a lot of this data now is declarative you declare what you want the environment to look like so you have a definition of what that environment should look like now after you deploy it right uh you you might have slightly different settings so the the definition uh could be based on a um on a number of things that actually get resolved in the environment it's in so you might have variables that get picked up in the environment that it's in so the deployed settings might be an iteration of the definition right an instance of the definition so you have to record the deployed settings you also might have the discovered settings so maybe after it's running for a couple of weeks uh the settings change and you need to be able to track that an iteration of of you know what what was it two weeks ago as it is compared to now extremely valuable to be able to track that over time because sometimes things deviate and it's a bad thing but sometimes deviations is just is just normal operations you know uh you know as as best practices you deviate evolve uh those deviations need to be recorded back into the into the Baseline they need to become the best practice in some situations so you need to be able to attract those deviations and and then of course over time you're going to have you know multiple sets of of of what we call snapshots this is the you need to be able to track this over time and this is you know one of the capabilities of devops config and and we'll go into that uh in detail so uh first I'm going to explain how it works right so we we basically uh get the configuration datings from from the settings from the source and typically today in in a modern application these settings sit in in git repos or GitHub and how do we get it well when you have a pipeline that deploys the application we could augment that pipeline right we can add additional uh commands to the pipeline to say hey let's take the configuration settings as they've been declared and upload that to service now and create a snapshot right uh and when we do that we actually create uh in an automated way a data model and the data model can be organized in such a way that it's business application specific it rolls up to a business application in the cmbb so we we know that uh you know specific types of config settings can you know as they change we can understand what business applications are impacted and so on and so forth so all the uh value you get out of the cmdb can now uh be uh basically analyzed at a very low level when I say low level I mean the actual key value pairs that make up a a config setting can roll up to that business service okay once it's in the cmdb we could provide various levels of versioning snotting as well as Access Control at a granular level a lot more granular than you would get in just a git repo right so the data is now sitting in the service now uh what we call the config data model and we could lock down we can encrypt it we can encrypt maybe specific key value pairs if it's a password or a a certificate uh key or something like that we'd want to be able to encrypted at rest so we have that capability as well so as we uh we start building up um you know snapshots of all this we also have a validation engine a rules engine that says hey uh these settings in this environment are either valid or they're not it's pass or fail uh so what that means is if you do have a config setting in development like let's say you leave debug turned on in development but you forget to turn it off when you're check it it to get and now we're going to promote it to production we'll catch that in the pipeline right we know that debug should not be turned on in production we know that uh specific uh you know the the passwords need to be encrypted in production we know that in development the the connection strings need to be different between all the different environment so these are the type of validation moves we can Implement to make sure that these settings uh as they move through the devops pipelines are actually valid okay so validation is something we're doing continuously for every single pipeline run uh and it's also a way to uh take snapshots so as the pipeline runs if if everything looks good we'll take a snapshot and now that snapshot becomes uh you know an immutable uh record of what was actually deployed in a specific environment right now once the data is available inside the platform you could also consume it by various you know provisioning tools right what that means is you might import data from anible or from Chef or puppet and then you can then use the same settings in a terraform template you can export the data in whatever format you might need to consume it so we normalize the data in the platform and then we have uh uh an engine that you can use to generate config files to actually generate config files in real time for the application for the uh environment and for the correct place you're going to deploy it okay so it's a very flexible and Powerful platform in that regard the other aspect of this is root course analysis okay being able to understand uh changes and snapshotting of changes now gives us um you know direct links into service now for example Incident Management I'm going to show you in the demonstration how we integrate with the uh service operations workspace for example uh for an alert and troubleshooting an alert uh and also GOC uh governance risk and compliance we can actually become the control point for a uh a GRC uh control right so so we can actually be the enforcer of that control Point meaning if the control Point says uh you have to have uh domains that are within a certain range or if you're sending emails to specific domains for your for your messaging uh we can then make sure those domains match with what the policy is right so that's just an example and we go into that in detail uh so what I'm going to do is go through a demonstration where we uh process incoming data from a from a pipeline we apply policy to it uh we validate it continuously so as pipelines run we're we're making sure it's always valid we're taking snapshots of it and then we're exporting it for consumption so that's what you're going to see in the demo but I'm also going to show you the integration with uh service operations workspace as well as the governance risk and compliance how how we doing uh Richard any any anything in the chat any questions not so far Joe no okay great let's uh maybe let's start with a poll and we can see uh just to understand today uh what folks are doing to manage their uh centralized config changes yeah yeah we want to get an idea of people are doing something today to help control the way people are making configuration changes or keep track of the way people are doing that it's pretty simple here a lot of people using get for this kind of thing um or some other kind of manual tracking uh obviously a lot of things happen in the tools that uh are out there already Network system tools have their own tracking tools as well but uh interest in what you have there so and and feel free to add anything into the comments if you can if there are other things that we're not covering here that you want to mention as well so and it's given another few seconds for a few more answers and couple more now Rich you can hear me okay I see one of the comments yes yeah yeah I think one person had some trouble but I think in general we're good for audio I could do the first 15 minutes over if that would the cas all right yeah wouldn't be the first one okay all right so okay okay we've got some good responses in the poll I think we can uh I'm not sure we it displays once we end the poll right Joe I believe that's so go ahead and uh if you want to end it then okay okay so now the poll results should be displayed so it seems like you know as I was saying a lot of folks are using git repos for their configuration set a big um and this is what we understand to be the case you know and it's been like this for a few years now uh and and what I'll show you in the demonstration is that's okay we can keep on using git repos uh but in a passive way we can start collecting the data we need from those git repos yeah and you want to say a bit more Joe about why we think we do things a little better than like you said we can continue to use git repos but uh I I think there are like security considerations and access considerations and things like that that are challenging and a get yeah I mean one of the things I think uh so get you know it's it's kind of an all in nothing Access Control right you you if you have access to a repo you basically get read and write access to all the files uh in that repo whereas with our solution we can be pretty gram about who can access what uh and and and and that's important you know U but it it's not it's not the first thing you have to do is the first thing I think it's probably best is just to leave things where they are and then allow us to passively start creating the data model and taking snapshots um the other thing is it can become compelling for for developers to start using all platform even if you're loading the config changes from uh from git they'll see uh immediate value if they start using service now to actually make changes to the config because they'll get immediate feedback as to whether something fails policy if something's invalid they won't have to wait for a Jenkins pipeline to fail if it's invalid they'll see immediately if something that's invalid uh it'll show them that when they make that change yeah okay so I think um we have one more poll and I'll go ahead and launch that and this is about you know how prolific uh infrastructure as code is right you say you utilizing infrastructures code practices today it'll be interesting to see what the spread is here because uh yeah people talk about infrastructures code a lot but we're not sure about how many people actually really doing it properly today so okay fair number of answers let's wait a few seconds more for others to add um there's a question here actually about this from from somebody's well if you mean infrastructure code do you mean AWS resources yeah AWS can certainly be defined with with code right so AWS has cloud formation built into it when which is considered infrastructures code so uh cloud formation templates uh you could also be using something like terraform or kubernetes that actually deploy into AWS but you define what you're going to deploy uh essentially in yamamo files which uh which you know are using templates uh and so on yeah L had another question too which I think is relevant to do here so um we we'll let's let's tackle this while people are still answering the poll so he as how does csdm 40 relate to this discussion since will it will introduce the build domain which it would include some elements although we're not sure which yet for sdlc I think that would include configuration from Central repositories if using the devops application does that make sense yeah yeah no that's absolutely correct so what we introduced I think in in CD csdm 3.0 was the concept of the uh of this the life cycle model and what we've done and probably it's a good time toh switch over to the demo now because I I'll basically start with this uh what we've done is we've linked the um the config data and the config data model to that to that sdlc model right and and that's just going to become uh more enriched as as we move forward with the uh with the c SDM yeah uh so that link is extremely valuable uh it basically allows us to use policy in the um you know policy that that can reference attributes of of that application service so maybe I have an application that's in the EU so the attributes going to say okay it's in Frankfurt and maybe it's going to tell me that it's customer facing so then I know that I have to apply you know gdpr governance to it and what that means is I have to make sure uh we're not uh making changes that will deploy a database outside the EU for example that's that's just one example right and and um you know so being able to have that information as part of the policy is extremely powerful okay great example because that's a difficult thing to do otherwise I would say yeah you you're relying on attestation you know now you're relying on attestation somebody basically checks a box saying yeah I did this correctly and what we're doing is we actually have the data to say you did it correctly or you didn't right so so here is a uh devops pip sorry Joe just before you go on I we still have the poll showing so just we'll just mention that before we stop it sharing but yeah as we'd expect actually not a lot of people yet would say they're doing infrastructure as code about 10% of those in the call right now kind of surprised the plan two numbers are as low as they are yeah yeah think the plan two numbers would be higher but this is a group that likes things the way they are stability and one more follow-up question here would that not make changing uh make sorry would that not make changes part mean you have to use the application Service as the CI of a change you can use the business service as well um so either of those two I mean there is some room for customization there so uh but basically we don't want to get any lower than than the application service so Business Service application service that's the model that's where we want to link the configuration data in the model anything lower than that like the components of an application that will actually be for infrastructures code that would actually be in the config data model because it's more Dynamic you could have 10 of them one day five of them the next they're going to get destroyed and recreated every time the pipeline runs so that type of uh ephemeral environment with the mutable infrastructure that gets created every time that's where that's where our solution is really is really going to be a good okay thanks okay so I I think we got this laid out uh you know we set the context here so what I'm showing you here is a Jenkins Pipeline and what you see is what we've done is we've augmented the pipeline with a a parallel branch that as you're validating the application codes what we're doing is we're uploading the uh config data basically we're finding all the Json files the ammo files the in files and then we're validating it and then we're publishing it as a snapshot and then we're using that published and validated and trusted config data secure config data to actually provision the environment using the expor generating the config files to actually provision the environment right so if for example something were to fail uh let's say uh well this case the upload fail or the validation fail uh and uh and and this is the example that I gave earlier if we found out that um we were provisioning we did an AWS region check and we were provisioning out to the US instead of the EU um we can call that out and we can stop the pipeline and we could prevent that change from making its way in production so uh as a developer I would see this in my Pipeline and I would realize you know this is like a quality gate on the pipeline not so much that the configuration is wrong because this would provision just fine and it wouldn't break anything however it does break uh the control it does break the government so if we if we move over to um if we move back to if we look back at the pipeline you know the first step we do is we upload the data right so we build the data model then we validate so let let me show you what that looks like uh inside of uh the application so uh I'm going to show you real quick the the config data in git because that's where it comes from right you know changes are made to get uh this is my cloud formation template uh this is where the E this is where the EU setting uh has been changed from EU to us but but we didn't allow it to get deployed right so this is the actual file inside of uh service now it's going to be represented here as a component okay so what that means is we we we load it up as a component CFT the cloud form template even though within this um component we have other details like uh uh data sources um instances uh AWS uh template information Rd sdbs S3 buckets you know everything we need to deploy the application all the data is here inside uh the cloud formation template including those database settings that I had where uh we made some changes to uh the region right so what I can do now is I can go ahead and make a change right if I go ahead and say okay let's go ahead and change this to uh Us East instead of instead of EU Central yeah I'm going to commit that change and then I'm going to kick off another pipeline um you know we would expect that to be quit now we have two options right we have two options we can fix it and redeploy is what I did between 623 and 624 I actually fixed it and I said okay let's redeploy it correctly but the other option we have is to go into uh in from service now uh from inside uh when the snapshot is being validated we can see here in that environment where the snapshot failed right I could actually find the actual policy that failed and there's quite a few policies here um I'm looking at the validation results of a particular snapshot and I can see that okay we're also checking to see that there's no white space and in places making sure the certificates are valid we're also making sure that they're making connections with https instead of HTTP all the URLs must use certificates and SSL protocols right so these are the kind of validation checks we're doing these are policies we give you out of the box essentially that that do these generic things you could also create your own policies right but when you do get an exception you do get a failure I have an option to actually request an exception so what this does and this is managed by uh governance risk and control this will actually go into GRC and create an exception and and allow that policy then to continue but in many ways it's um so here's the validation so so we provide that information back to um back to the developers they'll know when something fails you know why it failed uh they'll actually see it as a test result right so so this information is provided for them and then in service now in the um compliance workspace if I'm looking at the controls for let's say in this case it's gdpr compliance right I can say hey uh show me you know what the um let's look at a control objective I'm going to filter if you're using maintain gdbr compliance I could actually link those policies to the controls right so here we see uh devop config policies actually link to the control itself so there's this case the control is executed you know in 79 different places different environments and every place that runs we're going to run this policy to make sure that we're not provisioning databases outside of the EU all right make sense any any questions come up about that okay so one obvious question is how are these policies written right so how are these policies written well the policies themselves uh are written in JavaScript okay so within service now we have an engine for uh executing policies it's called the the policy as code engine and this is a new component it's like a new common service and service now that could actually uh be be used by other tools you know in the future but right now we're using it in devops config uh all the policies we run will basically uh run a little bit of JavaScript uh and this is what it looks like and you can actually take an existing JavaScript and tweak it to run your own policies so it's it's it's not too difficult to create your own policies especially especially since we give you all these examples okay so now that you understand we have uh quite a few policies out of the box right and and I'll show you what they look like here uh that you can start with uh you know it's easy to get started just by connecting to your pipelines and taking snapshots of the configuration settings okay so now we have snapshots and snapshots actually uh become useful because uh they become part of the change record so you if you saw the previous uh devops change velocity webinar you'll know that when we get to a production deployment we'll actually open up a change record right and what that means is um is uh we can we can see let me go to uh show you what that change record looks like we can see all of the devops data that came came from the pipeline all the commits all the work items uh all of the um the various testing that was done in the pipeline and if I look at one of these for example I also see whether the config data is validated so now we have a checklist of policies and we use the change approval policies here right to actually check to see that we you know operational stability looks good everything looks good and we could automatically approve this change based on that so we actually link the change the config change I just made the snapshot of the config data I just made to that change request and we see if it passed validation or not and if it passes validation that becomes one of the items on the long list of items that could fed into the change approval policies so we could automatically approve this right so Auto approval is a big part of being able to govern devops changes without slowing down the pipeline but without giving up all the different rules that you have in place before something goes into production okay and the devops config data is part of that check now the other thing I wanted to show you is that once we have it in place um once we have these snapshots in place uh we can now use it as uh as insights and an alert right so basically when you when an alert comes in an incident comes in uh you know in this case I get an incident and I'm you know I'm doing Incident Management now uh the alert comes in I can see no payments process for the last 15 minutes one of the things we provide with the service operation workspace is possible root causes and one of what that means is we're going to give you a list of all the changes that took place uh in the past you know let's say two weeks or so like what's what's what changed caused this problem and based on you know a little bit of some some uh a little AI in there and some algorithms they're determining you know they have a scoring system of which change most likely led to this alert right and if we look at the change request itself in service operations work space uh I can see the actual config changes so first I can see the actual uh config data model that's going of that that shows up here so I can go into the actual snapshot and see all the data in all the settings for that particular maybe I'm looking for a specific setting maybe I have some awareness as to what could have possibly caused this issue and I can see the actual values uh but more than likely you're just going to want to see show me everything that changed from the previous snapshot right and what we could do is we we don't have to go back to the previous snapshot let's say it's a performance degradation maue I can say show me what started when performance started deg grading two weeks ago show me that snapshot and compare it to the one today and let's see specifically uh only the things that changed and I see okay a firewall rule changed here right and that firewall rule could have impacted the performance of the application so I could drill down to a specific change from any of the snapshots that uh take place if I want to go back two weeks um I can see that there were some snapshots taken here on the timeline some snap shots taken here and all of this happened just before the alert came in so being able to drift down to specific changes while I'm troubleshooting an incident uh actual key value paay changes right uh is extremely valuable because this is something that you kind of uh you kind of lose in in the in the devops world you lose this visibility you lose this transparency when things are happening on the development side unless you start taking snapshots and and bringing them into service now yeah all right how we doing Rich any questions here no new questions no I think uh the audience is stunned at how good that is it's kind of impressive to see all of that the ability to go and just find that U without having to go and talk to a developer or start digging through code or things like that is uh is I'm I'm impressed every time I see it but uh no new questions at this point okay great so let me um I just want to show one more thing in the demonstration and that is uh back in the uh devops uh config workspace here uh we talked about the ability to basically use the data that we have in the data model to provision use it for provision use it for provisioning environments and this capability is uh is the exporters so what that means is um you might have some uh some different Tools in your environment uh that you use to provision maybe it's anible maybe it's terraform maybe it's Chef uh so you we can gener uh data you know for those tools we could generate data from the snapshots right so the snapshots are normalized data not not no longer do we regard the source of the data as the way it's formatted uh but what I can do is I can say okay let you know let's generate the the data in different formats so that's what I'm going to show you now um this is just a very simple exporter that takes all of the data in a snapshot let's say I want the uh the latest snapshot from that production environment and if I click on evaluate this will be an adjacent format it's going to generate the data in Json right so this is something I would use to uh um you know to to just see what everything in the snapshot looks like now I could also say give me the same data in a yaml format right and then I'll evaluate it and it'll give it to me in yaml format now what that means is the same exact data can be used by different tools right now usually you wouldn't generate all the data in a snapshot right more than likely um what you're going to do is you're going to say okay give me all of the uh data for a specific Tool uh I I like to use terraform as an example because terraform is is quite common but it also has a proprietary data format when you pass data to a terraform template you got to pass it in this tfv format now tfv uh looks like J but it's not really Jason it has it doesn't have commas between the fields so I can say okay uh give me data for a specific uh let's say a database so I'm going to say okay give me my DB and let's generate that now I'm going to generate at this time in a raw format because I want the exporter here to manipulate the data and generate it in the TF bars format and our value and this is the proprietary format so even if you're using like a homegrown tool and you want the data like in a CSV file or you want the data you want to load it into a database or you want the data in uh you know something that's more like a just a list of settings or or you know something that's tab Del limited you can use the exporter to generate the data that way now once you have the exporter in place you need to then be able to generate this you know whatever API you're using whatever tool you're using um you need to be able to get the data out so we provide okay if you if you're using service now or if you're using a curl command from the command line a lot of tools just rely on command line uh and use a curl command I could generate it in uh Python and it'll actually give me the actual code snippet to put in my python script to make the API call to get the uh to get the exported file so this is very powerful uh and it's very um accessible so now you know I have an endpoint that says this is my trusted validated configuration data and I have a mechanism to get that data in a secure way and then create the file and often I don't need that file to exist anywhere so if I start up my micros service like a like a spring boot server I can then delete that file as part of the startup procedure and then and I no longer will have sensitive data at risk on the file system it's all secured and service now so this is another way uh we can you know we can enhance the security of an environment okay so that is pretty much all I had for devop conf um I think we want to give the opportunity for uh questions yeah feel free everybody to put questions into the Q&A or any other comments you might want to put into the chat uh the other thing I was going to mention Joe and this might not be clear to everybody uh before they got on the call uh although I think it's publicized in various places but this all this capability that Joe has just showed you as this devops config what we're calling devops config it's a part as of the Tokyo release it's a part of itm professional and Enterprise so if you already have uh licensing for itm professional Enterprise you have this capability and we'd certainly encourage you to experiment with that um as well as what we would call devops uh change velocity which is uh kind of applying the same kind of policies to automating change requests from a devops pipeline uh really to help your devops teams as Joe said earlier to keep keep moving at high speed without having to worry so much about the governance pieces they never have to log into service now create change requests or or or worry about configuration policy we're handl this as a part of their Pipeline and you'd have entitlement to this again if you have itm Pro or Enterprise already today I was add that as commentary I don't see any new questions coming in Joe there's anything else you wanted to use to wrap up but uh this recording will be available there is a link in the chat already to to go and get that um as well as other resources that you can go and find out more about uh itm Pro and chat in the community with others about what they're doing itm I really want to say thank you to Joe for uh for putting this together and sharing that that in depth demo and thank you everyone for joining uh any last comments Joe no just want to say thank you as well thank you Richard thank you for the team here for supporting us and uh thank you for everybody that attended