foreign I'm Scott Murphy I'm an architect here at God past and I've just started a series on servicenow AWS Discovery this is part two of that series last Friday I did a part one which was covering how to set up a mid-server in AWS and then set up Discovery schedule so that you can discover everything inside of AWS well not everything but everything that it initially will let you do so there's a couple other things I'm going to cover this week I'm going to cover how to set up AWS config to the iubs config Advanced to update the cmdb automatically so it kind of negates the reason for setting up the schedule I had last week because it will automatically have AWS send config item changes so I had that set up last week in the environment I didn't realize it was running so it's still set up I'm gonna show you how to set it up and I'm going to take it apart as I'm doing that because there's nothing to really show if I just show you what I've already built so what I've already built I'm going to take apart and then rebuild it let me get to the demo section so I will start by showing the agenda first we'll do an overview of what AWS config is then we'll set up servicenow piece of this and then we'll go into AWS configure it to send events to servicenow after that whenever I initially did this I had to configure the AWS event handler in servicenow to handle the payload I think originally when this was all built you didn't have to touch the AWS event handler but in my current case I've had to do that so I'll show how to do that and I'll show you the error that I get before that happens so there we go and then after all this is done we'll have questions and anytime anyone has a question raise your hand I it's hard for me to do the PowerPoint and all of this so if someone could speak up feel free to interrupt me so what is AWS config it's a part of AWS that can raise events and send through rest an event over to an instance or anywhere really but we're going to configure it to send events to servicenow instance my personal instance and then you're going to have our cmdb items that it's discovering show up real time with changes real time is going to be about five minute delay on whatever you make a change over there it sends the message pretty much immediately but not you know necessarily as fast as you'd want then service now has to trigger a discovery and go across I'll show that in the next slide I've also included a link to the AWS config I'll show those websites whenever I do this so the servicenow documentation has a lot of stuff in it I'm going to show you kind of a smaller version of what they show they have some extra things they've added since I did this the Utah documentation has some extra properties that I didn't even know about until I looked at it yesterday so but it's really stuff you don't have to know to set it up it's just extra things so I'll cover what those are but after I when we get into the demo setup here is a Visio diagram of how this works so like I was kind of started explaining earlier AWS config service can send events to something we're going to set it up to send a rest message to this scripted rest API and servicenow the scripted rest API takes all the events and puts them In This Cloud events table and then there's a cloud event scheduler that will run based on this and it will do a seem to be update it'll go trigger a targeted cmdb Discovery to that cloud resource I guess I should say whatever it is that's getting updated and it will update the CMD if you miss that so those two extra things that you that was talking about the system properties that I didn't know about and until yeah just the other day they're they're mentioned on this video as well so there's a cloud event scheduler that you can set up to do parallel so that I guess if you're getting a lot of configuration item changes in your environment you would want to set this up to do parallel I didn't have the need for that in my environment it's just a simple PDI and I'll just be doing and change at a time so it's not going to slow down the system the other property is the use response mapping I don't know what that is we'll have to look at the documentation on that one and so this I wanted to lay out how the events are processed AWS sends an event servicenow this is just the word version instead of the picture sends it to this scripted rest API once an awf submit is received from the cloud it it's inserts into the cloud event table then the cloud event handler triggers this is the piece that I mentioned earlier in the agenda that we'll have probably have to modify um using the script include regular record the event information gets processed by the script include in the Handler script it will then go trigger that targeted Discovery I'll go through that script include whenever we get to that point demo time okay it's in this PowerPoint and get into showing you how this works and how to set it up after the videos posted we'll get a knowledge share I'll probably create a knowledge article and put these documents on there I didn't do that for the one last week yet I never made a document for that so I'm going to go do that so we have a document for the mid-server piece Emily had some documents that she was going to share with me we're going to get together and we'll make a knowledge article to attach to the first year of the wealth so first we're going to go talk about the components needed to send events it's one of the steps we're going to perform really the only thing you have to do is have a user that can do this so I'm going to go there's my user so you have to set up a user that has the ability to do this I am that user I logged in as that user to change the password let me get back to my admin user the history here so you just set up a user it could be anything you want since in the future I'm going to be building multiple Integrations not just with AWS but with Azure Google G Suite things like that I've made a group called Cloud Integrations and I've tied the role that you need to that for this one specifically is the s in underscore CMP Cloud event integration role we're going to use this user for the Amazon AWS config service to send it's the authentication for it so that's going to be in one of the early steps of the setup so you have to have a user and it has to have the roll cloud event integration in order to send events to the script address API okay so back to the document here's the components needed to send events so you have to set up an Amazon S3 bucket a simple notification service topic and subscription this is the script subscription it has the username and password that I just showed you set up AWS config and then you set up an event Bridge Rule and an event bus the NBA bus has to be turned on to send those messages so I will cover all of those things in AWS and we'll start with this one so first one is create a bucket in the region that you want to record events and then you create a topic with that type of with the type of standard so I have a screenshot here in this document I'm just going to show you the tool what I'm talking about there so S3 so log into my AWS and I'm going to the service S3 scalable storage so what I'm doing by your region you have to have a bucket set up in the region that you're your devices are in so if you did have devices in several different regions you have to have a bucket for each or I haven't done this yet but you can set up some kind of aggregator that can aggregate everything from all your regions not that high speed on this yet I just know how to do it for one region at a time so that could be a later share of the wheels whenever I get to that point to see how that works I've got two buckets already like I said I've already built this so I'm going to delete these oh my goodness I have to type in the name of it screen there we go okay I'll leave that one alone I'll just go create a new one create a new bucket they have to be unique I guess across everywhere not just your environment which is kind of weird but that's the way they have to be I'm going to set this up with a region North Virginia USA East if I don't have any devices in there I'll add some and that'll be part of the demo you've seen it work okay next acl's disabled I think I just went with all of the settings that came out of box with that double check create a bucket in the rookie I didn't even show it a screenshot of that so yeah I think I did just use default settings there so we'll have this new bucket created so that was pretty simple well you just name it basically and then click next Next Step would be to create a SNS service topic so I'm going to go over to that now and that is a service over here called Simple notification service here's my topics that I already have in here I'll go ahead and create a new one actually let me turn these off because they may interfere with me doing this because every sending messages too so let's create a topic you get two choices first in first out or standard for this purpose we're gonna do standard it allows for the subscription protocol HTTP which is we're going to be using and then you name your topic you can give it additional name if you want other than that you know it's that's it you just name it make it a standard and that's it I never changed any of these other things next we need to create a subscription and we'll do that we are going to use https we're going to set their endpoint and that is a URL that I hope I have in here that I could paste in this is what the URL should look like the https your username that you're going to use colon the password you're going to use then your instance name and then API now Cloud again and that should be it you said all those things added anything else for that so you just pick your topic or your topic that you just created it's gonna you did that already https and then you're in create the subscription so when you do that it immediately sends out an API call to your instance to get validated so we're going to give it 10 seconds or so and we're going to refresh it the status is now pending confirmation it will change to something else whenever it works or if it doesn't perfect it didn't work so we can look at Cloud events over here to see if that came through we did not because these generated passwords have funky characters in them the whenever you put that in that URL they don't work so I have to log in to that user I'm kind of glad this happened because I had to show you this okay back over here do this again oh yeah Chris asked in order to set up the subscription to the event do you need to verify that on your endpoint yes it will automatically do that that's what we're trying to get okay it works so I must have fat fingered the password earlier when I reset it before the call so yeah now I'm gonna go show it automatically replied and accepted that confirmation so Chris you were right that is what was happening behind the scenes now I should be able to go to my cloud event and see the cloud of the subscription confirmation so there we go this is the payload we received and whenever this payload was received the event handler that I was talking about earlier grabbed this I went through that script include and I'll cover that at the end I'll show you the logic for this and the logic for the configuration item changes that come through but basically this payload came in and then it recognized it and sent the confirmation back to AWS confirming the subscription but that's done automatically so it's not something you have to click on and do you just have to have the right password for the user and the permissions I demonstrated that well uh okay so that's that next step is configured for it so it can now send events to us successfully now we have to configure an AWS the events to be sent so what do we want to send there's tons of things you can send from AWS we're just going to focus on configuration item changes that's all we care about in this scope the next thing we're going to do is go into AWS config and use the bucket from step one to top it from step two and you should do this all in the same region that we set all this up in before so I was in this is in go back Ohio East U.S East Ohio so we're gonna go over here and make sure when I go to config I'm calling AWS config but its name is just config so that can be confusing to someone we go to settings okay by default this is off um you have to click edit enable recording and then you set all the things in here to what we had earlier so let's start from the top I don't think any of this was changed no that's retention of how long you want the messages to be kept here you're going to choose a bucket from your account I made that new one so we're going to use that new bucket and strings regression changes to negative choose a topic from another account yes it may be from another account SNS topic on oh okay we're going to choose a topic for my account that's the one I just created share the most demo then if you'll see at the bottom The Next Step you need to configure a AWS Cloud watch event rule well that changed when you go click on this and select no it's eventbridge is what it's called now so that's the next step we'll have to do that next so I'm going to save this and the whole purpose of this was to turn on the recorder to record these messages and you're telling it where it wants to send them and because I can't yeah okay to the bucket so I must have messed up my bucket configuration see it says they're not submit sufficient privileges on the bucket so I must have skipped over something there let's look over here and permissions block all public access and that it has something to do with that we'll turn that off let's go back to config names I have to edit this and pick our new bucket and our new topic oh man okay so same thing you know I do what I just did there the other bucket had it on so let me look down oh there's no policy here we go that was it I'm gonna set the policy there I think whenever you do this the first time it automatically generates this policy yeah yeah it does so I have to figure out the number okay there we go so that was little stuff that I had to do I don't have any of that documented I never had to do that before I think because the first time you do it like I said it it takes care of that for you when you create a second one you do have to do that so that was adding permissions this is AWS code stuff uh I don't really know I can't explain it very well other than to say that I mean it's you're configuring a resource and telling it which what it can access I don't even know what language this is exactly yeah well it's Json but it's yeah it's it's configuring the permissions that it has now let's go see if it worked I'm pretty sure it did because I kind of remember having to do this when I made my second bucket earlier but I didn't document it so I'll put that in my document next for troubleshooting steps so where was I I was in config I was just trying to change it to the new bucket and topic oh he said region yeah we figured out so sorry I was late to that response okay so we go to settings and go to edit and we want to change it to two our new bucket our new topic and it should save yay we can move on okay so we've set up the bucket we've set up the topic and subscription to the instance we've set at a dose config to record the events into the bucket and for that topic um next is the event Bridge rule so you go to find a rule in the event Bridge so let's go over there to that Amazon event Bridge and then over here we go to rules I've already got some in here I think this was my original one let's burn it down these other things are system created I'm not sure exactly what they are for but they were there whenever I got here okay we're gonna use default here this is we're going to define the pattern for this uh let's see we'll intervent pattern yes okay we'll leave that alone up there this is optional this just kind of lets you see the Json response so what it's going to look like we want config item change that's the event I don't think you have to necessarily pick this but this is just a sample event so you can see it so this is the structure with which we're going to get messages whenever a configuration change comes in part of the reason the way I knew to fix the AWS event handler and servicenow was the payload that we're receiving didn't match how the script include was expecting the payload to be so that's why I had to go change that and I'll detail that a little bit later but this is was the key uh you know the payload that we're getting this is the way it's expected to come in it'll be sent to us from you can copy it if you want look at it in a another way if you want to so next we're going to pick the This Is Us picking how we want to get it so that's basically we just saw a sample of it there we go so this is it telling it that's what it wants to use and that is it next Target should be AWS service you pick your topic okay I'm gonna pick the topic demo next configure tags I didn't make one and there we go we review everything that we just did and create the rule Bingo you need to go make sure your event bus is activated start discovery will activate it so yeah so again buses there's just a default one that was already here I didn't create this thing so you go to it and I already have it running because I've already set it up but originally it's in this state here not initiated you just click Start Discovery to make it start doing that okay I think we're ready to show you how it works I have a link here instructions how to set up AWS oh I do okay good I have documentation that shows you how to do this across regions so it's actually from Amazon so they've got a method where you can set it up across many regions you I wouldn't want to have to do this for every region that I have devices in right to keep it simple and share the wealth I just want to cover one but in reality the client is going to want to have do this one time and kind of share it across all the regions and have them all filmed to the same instance probably so any reason they wouldn't want to do that is if they had multiple instances of service now which would be crazy and now I can go into servicenow slash AWS and and make changes to things or make new things and then there we go say look uh there's some of the stuff I just did showing up so we're getting messages for this new setup so I'm gonna go show me creating a nice thing let's go do a server okay I don't have any instances running still in this yeah Ohio okay let's create an instance we'll create a Linux we don't have to log into it I don't so it doesn't really matter just something making sure I'm doing free tier eligible stuff pretty much whatever is already there you don't have to change anything should be good bounce back and forth between servicenow and this to see when it actually sends messages you know I'm just realized my um my picks for the event handler is already in place so I'll go show you the original one versus mine so you can see the difference is to yep okay we're in the same region I'm thinking maybe I don't know I'm gonna just keep changing it to see if if you turn it off turn it on maybe it'll I thought creation went through maybe it does not um but I could have swore whenever I set all this up if you create something a creation of something shows up over there but maybe it's just changes and so this is a change me stopping it I know that for sure shows up maybe it was just delayed I can look at the message to see if it was crazy or not but here it is showing up in here oh so I already triggered a discovery so there's the instance right there I think this is the resource ID that is processing so that's running right now like I said it's not real real time it's just close especially when you pick that topic of standard it is you know first in first down is or no it's not first in first out what I'm talking about is the the speed at which Amazon will send it I think this was that that I was looking at standard is slower first in first out is quicker but it's different kind of message type so I've never tried that method I've only done the standard one so as it gets all of these messages and processes them it creates a little mini Discovery and tries to discover each little piece that they got so we should now be able to see okay yeah so scurry's finished this is the resource ID that came in it's the actual virtual server see how I made that ec2 instance these are all the pieces of it came in right there's all kinds of other things that go with it but here it is so I'll go to that configuration item and show you here it is so it is seen as the state is off I think it's because I turned it off while it was due to Discovery but we can re-trigger all of this again by me going over and turning it back on so I did want to show that too that was part of it I wanted to show that you change make a change it does do that so it is off we'll turn it on since that might take a few minutes I'll get into the event handler and show you the second piece of this so done with AWS config here's the AWS an event handler document okay first I'll talk through some of this I think I mentioned how they're processed before why does the aw stupid Handler need to be changed I'm going to cover that by showing you the two script includes the out of box one versus mine and then you know we'll go from there so go over actually I'm gonna go to it but I also want to go to the very beginning of this so I want to show you the scripted rest API the code for that so let's find the cloud one place so this is the API that we configured um AWS to send events to so I just want to go into it show you some of the code inside of there just so you could follow along the path with the code not just the Vizio so there we go so this is showing when it gets a post gets the payload separates out some variables and then it creates a record into the SN AP Cloud event table which I was just looking at and refreshing so I think that was over here oh look see this is on now so that automatically happened it took over and now it's showing this is on so yeah so this is the cloud event table right here so this is where all the events populate so AWS sends it descriptive wrist IPI processes it puts it there I don't know what causes it to use the event handler I haven't seen any unless it's embedded somewhere I haven't seen a flow I haven't seen a business rule haven't been able to track that part down I just know that this event handler is what processes or handles anything that goes into that table I thought you know I could be able to track it down by looking at business rules or something else like that but so far I haven't and I just know that it works so I know this thing processes it so this is here um this is an out of boxing that was here it just had a different Handler script so the other script I think was just AWC Andreas AWS event handler oh I made a okay I made a different one okay so that was what was originally here was AWS event handler so depending on your payload this is something you should be able to configure for yourself right because you might have different payloads a reason to have a payload different than what servicenow is expecting in this case servicenow just didn't have the right it didn't support the way configuration item change was out of the box and so I can open up that script to include open up the Handler custom one and kind of explain the differences it'll be super technical but we're going to go down that road it is something I had to do and so you do all the stuff I just did and it didn't work the AWS event handler will be the issue if you see events that are they're coming in and they don't work that could be one and so I guess what I could do to show that is to change it back to the original show you the area that you do get clicking around a lot let me find where I was the do that we're going to trigger this thing to turn off stop okay so that's trigger another message come through while that's coming through and not processing you should come in here and give an error whenever it gets that we'll have that to look at so where was I going script includes okay let me open that up over here yeah okay so this I think basically only has one function and a couple sub functions yeah okay so there's a process event first thing it's going to do it kind of would help but if I look at the Json that we're getting so I'll pull up a Json but the first thing it's going to do is come in here and start parsing that Json checking the account cloud account making sure it is a cloud account that you actually own so last week we set up the cloud account in the system gonna go validate that that is a true cloud account in your system so that's what it's doing here at the very beginning then it looks at the subject and type of the message and figures out if it's a subscription confirmation remember we talked about that earlier this is where it has the auto subscribe functionality for the subscription confirmation again this is all out of the box This Is It Was Written already for us as part of the application so this part is like that and then here's the configuration item change type that's in the payload whenever we go inspect that payload that we're getting you'll see that in there and so I think where we started having issues was it right around here in these lines where it was looking at the message getting the configuration idle resource title A lot of these things is where it fails because the Json is missing a layer it's called detail so if I go over here to my modified one I'll look down through here and when I get back to configuration item change there line 72 yes so message dot detail configuration item so detail was not in there but it is a piece that's part of the payload so I'm going to show you the payload now these are messed up so let's look at them I'm going to grab this payload let's make it beautiful it's not beautiful this is not the one I usually use there oh I may have gotten the uh I did put it inside the brackets at the top yeah okay this is in a different type of Json format it's got the slashes it really threw me off when I first look data it made it really hard to read so I don't know this is another way of notating Json with these slashes and quotes I don't remember I just had to kind of look at it and see that whenever you got to the the layer in here for message messages body Dot message Chat someone said something can do a parse on that then a stringify with no slash T in the code yes okay yeah I don't know how to be to find out but let's try to find the body Dot message and the detail property so here's the message and then we have detail type the script is looking at things like this to determine how to process it I think that's what I want to do though Chris I might put it in a background script I may have done that whenever I looked at this I'm running out of time here so I do want to open up some some questions but basically took this jumbled mess and found out that there's a layer I think this is it right here it is a layer that I had to go put into that and so that's really it that's really all I did I just added that detail after message and that and I had to do that several places that you could see message and that's really a lot to change to make the event handler I do have the code that I changed right here and I gave instructions to the client with this document and said hey go make this custom script include and I gave them the Code here to put in there so I didn't actually do this for them but here's the instructions for that so I'll have this document out there if anyone needs to use that code you can so yeah that's pretty much it that's the last step and that was just me showing that you know the out-of-box one don't work the modification I made by adding the extra detail after message everywhere that that is that made it work so if anyone's ever tried this following the instructions that are all out there it probably failed right here just using the service Now instructions that got me to this point and then I had to go figure this part out well next week I'm going to show how to do kubernetes Discovery it's kind of very involved too it's it's a lot of you got to know what kubernetes is if you do great I'm gonna have a introduction to what it is what I understand it to be and then I'll show you how to set up service now to discover that it's a lot of setup in AWS so it's just probably very few things you have to do in service now to get it to work we've already done all the so all the work we've done Builds on the next thing so that's what I wanted to do with uh kubernetes it's just another layer of things that's why this series was one two and three like this so that I don't have to go repeat everything every time I do a shares will on AWS so if you've we're lost in this one go watch go back and watch share the wealth number one and you'll be able to understand a little bit more of the context of where I started off at and share the web number two and then the same thing for three I think three this this session doesn't really matter as much it's just an extension of one so it's not related to this piece at all but if you make changes in kubernetes it should show up because we did [Music] thank you