AWS Cloud Discovery (Part 1) | Share the Wealth
foreign TC here and I'm gonna hopefully show you guys how to install a mid-server on a Linux based ec2 instance so in order to install your mid server on your ec2 instance you do need a couple of things the first thing you'll need is a mid-server user with the mid server role and then the second thing you will need is your actual ec2 instance so I'm going to go ahead and spin up an ec2 instance really quickly for you guys just to show you so from my AWS console here in the ec2 dashboard I'm going to go to launch instance and then we're going to go through these steps ec2 instances are really customizable but for this we're going to stick to the default and free tier options so we're just going to call this share the walls mid instance I'm going to stick with the default AWS API here it is Linux based and it's like I said free tier eligible and it's perfect for what we're about to do all of this I'm going to keep the key pair would be used if we were going to be remotely accessing our instance but I'm going to connect directly through AWS so I'm just going to pick one I already have all of this looks fine so I'm going to launch my instance okay and now it's starting up so we'll give it a minute and as soon as it's done we will get started so now that we are up and running I'm going to hit connect so I'm connecting directly through AWS so like I said I'm not going to need that key pair um what I am going to do though is do the web get call and let me show you guys what I'm doing so from the mid server installation the downloads you would just copy this download link right here and then directly from your instance you're gonna do that so you're going to call basically or just pull this file and it's downloading right now and once it's done we'll go to the next step so the next step now that this is completed successfully and it saved our file we need to access it so I'm gonna use the pseudo command and and then I need to put the package name which is just this last part right here okay so it's going to verify and then continue with the installation okay so now our we've installed this it's been it's downloaded on our ec2 instance and this path right here is where it is located um to get there though when Scott and I were testing this a couple of weeks ago we ran into a problem and you need to switch to the root user in order to do that so we're going to do the pseudosu command here and now you can see we have switched to the root user so the next thing I'm going to do is I need to go up in the directory to see that file so I'm going to change directory twice and then I'm going to list out my files to make sure okay so yeah I can see everything and now I need to go into the opt folder here so yeah some people are asking questions about explanation of some of these commands so wget is a uh it retrieves contents from web servers so it's like uh clicking on the download link and it gets downloaded to your downloads folder in Windows or any other you know system Mac whatever whatever you used to that we wanted to show Linux because it's more of a niche type thing we Emily had a client that wanted to do theirs on um Linux so we had to go learn how to do that and so we wanted to show this one because it's kind of more complicated people aren't used to them businesses use them a lot I see a lot of people using Linux mid servers so we just want to show you how to do that and then the monitoring windows so I'm not going to show you how to install the modern windows but you know I use the windows mid server myself I took the easy way out but right families I just wanted to show you know how to do something that's kind of more complex I wanted you to know how to walk through this with a blindfold on so once we have our directory and we see and we can see our opt folder we're going to list all the files there so I'm going to use the ls-a and then we see this servicenow file so we need to navigate into that so I'm going to CD servicenow and then again ls-a this is just so you guys can see where everything is in these directories and that's our mid file so we need that to so I'm going to C D into the mid file and again a little repetitive here but okay you could definitely navigate straight here with one command a CD into it to go over to the file but you know she just wants to kind of illustrate you know where exactly a downloads to and in the document we're going to share with you guys after this we have everything explained and like what the commands mean why you have to use them but it is I mean if you you know want extra size or whatever you have to start over so it's yeah so now we're here and I want to launch this installer file so to do that I'm gonna just do okay so now we're going to run that all right so the next things you're going to need for this part are your instance URL so I've got that um when it asks you if you want to use proxy we were just answering no for the purpose of getting it installed and same thing here we answered no okay so now this is when earlier when I said you'll need your mid-server user this is why you'll need it because you this is how we're logging in right here so mine is mid oh wait it's server one and now my password so right here when if you type or just copy and paste your password like I just did that cursor is not going to move so just make sure right here my username and password are valid so I'm fine to move forward if for whatever reason you mistype it you'll have to restart and just run the installer again for this I'm going to answer no all right and now I need to name my mid server and provide a little bit more information so we'll call this one AWS Linux mid and then my unique name and then the long name okay so for this here um I don't have that much space but I'm going to install it anyway since that is you know it's the only thing we're using the server for and then my non-root user is just ec2-user let's see pretty short answered yes there last time okay I don't know that do we know what happens if you answer yes to some of those things I don't know Roxy and the mutual Authentication I'm not sure to be completely honest I hadn't played around with that yet so now um everything's done the mid server is installed and we see it running as a service so before I give it back to Scott I'm just going to quickly go into my servicenow instance and make sure of that so right here this is our AWS Linux mid I'm going to validate it really quick we'll save that and now my mid server is validated it is up and running and it is ready to be used I'm gonna go through how to set up Cloud provision and go well there's let me go first with the plugin so there's a plug-in that you that I'm there's many ways to do file Discovery I'm showing you how to do it with cloud provisioning and governance that's the method that I became familiar with there are a few other different ways that you can discover things in AWS I'm going to cover how to do it with Cloud provisioning governance first of all I was going to demonstrate the plugin that needs to be installed and it's called cloud spell provisioning yeah so this it's the cloud provisioning and governance core and that consists of two plugins Cloud config management and Cloud API so these are the two things that come with that and that's what's that's all you have to have installed I think in order to get this to work what that does give you is over here on the left hand side yeah there goes Cloud provisioning difference it gives you this module Cloud provisioning governments and Cloud API as well you don't have to use any of this up here I haven't I've just been using uh this section of it right here so to start off though we've already got the mid server installed I want to go back over into AWS and I'll try to explain as much as I can about AWS and how that kind of works uh first of all I want to mention that I'm using two accounts I've got a master account here and then I've got a what's known as an organizational account it's a member account is what we're going to call it I'll call it demo account one it just goes to show most clients have this type of hierarchy they have one main AWS account and then they have accounts underneath that so I didn't want to go with a simple you know scenario of just having one account because that's just not very real out there in the world I kind of started out that path with just my main account but then I was like oh Gartner has like 100 other sub accounts in their main account and the reason for that is you can set up the authentication with the master account and then you and servicenow you allow to assume a role into all of the other accounts so it's a way to set up one thing in service now and then it allows you to authenticate to every other account and discover anything in all the accounts so it's just a way to you know make it a lot easier so first I'm going to show what you have to do is you have to go into Services here and open IM that's where you manage the permissions I already have it up here in the purple window so the first thing you're going to do is we've got to create a role and so that role is going to be this servicenow ec2 rule and this is the role we're going to assign our mid server so I don't know if Emily she didn't necessarily have to show that part she just showed creating it well the next step you're going to do is make this roll and then apply this role to that server and you can kind of do that in in the opposite order if you wanted to if you come in here and do this first and then as you're building it select this role for that device to have it doesn't matter either way but the main things you have to get out of making this role is I'm going to make this a little bigger hopefully it helps I can't switch as easy across my screens but the main thing you're going to want to do is in here is set up the permissions read-only access is really all you need here and you need to have it also this trust this servicenow symbol that's the one I was talking about earlier where any you can assume that role and get into any other account and I'll get it I'll show that one in just a little bit the next thing is you want to have a trust relationship and this is kind of along the path of the API that it uses so it's going to use it needs access to allow ec2 amazon.com and zoom role into that and use basically use that service that's a API service and then later on I'll go over eks which is kind of kubernetes that's that's what I'll show later that's why this is in here it doesn't have to be in here if you just want to do ec2 servers ac2 is with a elastic cloud is what that stands for virtual machines this is what that you know really relates to so that's that so that's where you have to have this in here since I've already got it created I'm not showing how I created it uh if you need to see that I can do that later but that's the two things that matter here is the trust relationship and then these permissions uh you set up view read only or view I have both in here I really really you only need read only this one doesn't necessarily matter next I'll get into this servicenow assumed role so you create that here in the master account the servicenow syndrol and inside of that role you have to say that this right here is called Arn usually there's a account number right here and that account number you can see up here for this role is my master accounts account number what this is saying is any role with the name servicenow ECT rule is trusted to assume rule across the whole company because there's a star here so it lets you go anywhere in the whole system to any account so that's what that is saying and then I'll go into that role here it's pretty much going to show the same thing AWS organization actually read-only access is all you need this is this is what I'm using for kubernetes we can wear about that later this is for where your mid server is at so the serves now soon roll has to have the exact role that is for the ec2 server and so that online environment is sitting in my demo account my member account which is below that purple window that we're just in that other that master account so that role is the roles are different per account they can have the same name but that account number is what is different so I set up the same exact roles name wise in the other environment as well but they are separate you do have to make them in each place and so as you can see here this is the iron for the ECT role here my mid server I'll go ahead and show that that is in the ec2 section you can see here uh what instances you have and which ones are running but you also have different data centers so I've got to figure out where I put it there it is North Virginia so I've got a Linux mid also and I've got this is on Windows so here's my mid server I'm going to just go into it to show you where you set the I am roll there it is and so my main server has that role that's the role I was just showing so back to what I was talking about on the other window the zoom role has to trust the role for the ec2 ec2 server that you mid server that you have and you have multiple it's going to be the same role for each one of them but you only have to have a mid server in one of the member accounts you don't have to have a mid server in each account you can just have it in one as long as you set this part up this assumed Roll Part up correctly here in AWS that's what's going to set you up for success and this is was honestly one of the hardest parts about this for me to grasp when I first saw it so there's a video I did watch that explains it and I'll share that link with you maybe you'll understand them better than you understand me I don't know but so I just covered all the these accounts assigned it to the mint server now I think we can switch over to servicenow and show you the servicenow steps we've got this installed let's go over here and set up the cross assume role or you got to set up the account go to the service accounts and set up e-accounts of course already done this so this is where I was talking about the account ID that matters I'll go hand and do new that way we can see what the form looks like and really all you can you can name it whatever you want and then all you need is the account ID you get that from AWS and that would be this account ID here at the top right of your corner right corner you put that in there you do not have to put any credentials you do not have to put the URL all you have to do here is then select the data center type and so I'm covering AWS that's so obviously you picked the AWS data center it's generally the same concept for Azure that's what I'm probably going to go try to conquer next is that one um hope and maybe someone out there has already done the Azure they could show this to but for each one of these you'd have to go get an account you know in each one of them I'm on a free account right now with AWS so yeah so you pick another this data center type then it lets you pick whether or not this is a master account or not my master account is the one that has that so I'm going to switch back over to that and I'll just open up one of those records your sleeve so the master account will look like this so yeah you say yeah this is a master account and the assessor account is where you it's the account where the mid server is at so I made this super simple in mind I don't have more than one demo account or assessor account I've just got this one so that's where my mid server is physically built at and that's where you have to pick that so the master Whenever you set up Discovery you set it up on the master account you do not set it up on the demo account or the assessor account you only have to set it up in on the master account and it will go out and discover anything that is a child of the master account next I'll switch over to the demo account and just show you what that form looks like pretty much similar thing you just have a parent and you don't select Master you don't have to select assessor account because it is the assessor account so there we go once you do that The Next Step whenever you create these two accounts is you have to set up the crossed assume role so first I went into service accounts that's where I'm at now next you go down to AWS cross assume role parameters and this is very important too this is what is going to allow you to look into all the other accounts that are in the system so here you need to get the Arn access role name so that's the role name from think my master account let's go look and see yes so this is my master account and this is for the servicenow zoom role that I made it needs the the Arn for that so that's what you go over here you put here and then you say which account it's related to it's related to the master account this is the key in service now that lets it Go discover any other account under the master account so this is very important if you get this right okay once you have that set up you can go set up your Discovery schedule and start discovering I've already done that but I'll just go show how I set it up I usually just go to the master account you can go directly to Discovery schedules and do it you know there's other ways to do things in service now but logically I feel like this is the best way to do it you come here and you first would click discover data centers this is the first true test to see if it works at all just by doing that right there and while that's running I'll again explain this is credential risk Discovery so we're not we haven't put it if you have noticed we've put no credentials in here at any time you don't have to it's credentialless there we go we go back completed reload this form to see an updated data center list and so if I didn't have any in here this would you know have populated for the first time so it it populated all of those I did not get an error in this discovery status window so we're good to go we can move forward next you can click the create Discovery schedule and whenever you do that I've already got one so I'm just going to go through the one that I did set up so it should already pre-fill a lot of these things for you but we're going to say select account you only have to do the master account like I said you do have the choice of however many accounts that you went back and created earlier it'll also let me go back one step it'll it'll fill out all the sub accounts for you so not a big deal I've got two so I made that you always have to do the first two the assessor account and the parent account the main master account you have to do those two but let's say you have 10 more child accounts under the master you don't have to go create those manually you can simply in the master account click the related link refresh sub accounts and that will go load all of your sub accounts it'll run that pattern come back and fill all those out for you so just want to show that click that run it it's not going to find anything new because like I said I've already made it but I just wanted to go through that step there we go so they populated successfully and they should show here in the cloud service accounts there's all your sub accounts it was already there but hey if I had more I could do that back to the Discovery schedule Oh Omar you ask how I made the sub account okay that's a good thing to cover and then you got to know a little you got to go study a little bit of AWS to get how that stuff works it wasn't too hard to figure out what this is but it's cold it's under whenever you do make an account which is pretty straightforward you go to AWS make yourself an account you'll kind of be at this place right here I mean you won't be on this page exactly but you'll be able to go up here and create another organization and this is where you set up that second account you'll be able to go here and create another maybe I had to do it from now I don't remember exactly how I did it but this is the screen I did it in you have to have your own another email so I just made another Gmail you know it's it's also free you know for a year So eventually I have to redo these probably to keep being free but um so I did that I already had this down here and then I made this one that's my account I could go you know make up another email make another demo account and have more you know Children of the master but uh for now just for Simplicity that's the way I've done that so hopefully that answers your question you do it here my screen share box is covering it yeah you can click add an account here name email there you go so that's how you do that that's how you get more AWS accounts but that's like if you get an engagement with a client this is all done right you do not have to worry about you know the AWS side of it so much as it's good to get in here and kind of get familiar with it you don't have to be an expert at AWS you just kind of have to do these couple of things to make digs I don't understand exactly how all those roles and permissions work not a not great at that myself I was just able to kind of follow a video along follow a couple instructions and I did kind of poke around to understand how those exact roles work I think because of the Arns and things like that and an API that's all you really have to have to make this work you don't put any credentials in here at all we haven't had to do that because of the roles that we've set up in AWS it's allowing it to work okay so back to where I was at it and creating the schedule this is kind of really the last part of what I wanted to cover here today here it's already pre-selected this your account ID credentials you don't have to select anything like I said and you just pretty much have to give the Arn and then the account for Access that accessor role that all might be filled out for you I don't remember the first time you do it you might you might have to put this in there on your own but I've already got a schedule so that might be why this works and you can click test account if you want I think it already does it when you click it that's why it already says it's successful I didn't click it again and then down here you can say it shows you the demo accounts ready so we'll go to next schedule name already exists so I've already done this that's the schedule names over here I just want to go through this so you can see all the steps there's a good step that I like over here it shows you know how we discovered all those data centers that corresponds to I'll just pick this window here's all your data centers over here on the right so that's a good correlation there to what you're seeing in servicenow for my purposes it really speeds up Discovery if you only pick the ones where you have stuff now real world scenario they might have stuff everywhere I wouldn't suggest making this list smaller but for myself and me wanting it to be faster I remove things I can't um on one of the latest sessions with a service now on like item stuff they suggested that you remove pretty much everything except with the ones data uh regions that you actually have so that is yeah that's my suggestion too so yeah I will remove all of those I'll leave all the US ones I might put I don't know where I have stuff I'll put them into I do want to show you that there's you can discover in different regions data centers is what those are I do want to show that so I think I have things in a couple different places and then in this further along we go in the series I might pop up kubernetes in the west one cluster or something like that so I'll keep all of these in here it doesn't take too long to run through those four then you click you select your Mentor um I'm gonna just let it Auto Select I really only have two mint servers in here and only my main my windows mid server is up and running so there are some things there is capabilities that you do need to add to your mint server so let me go cover that real quick before we finish that thought those are not so there we go so I don't know clients do different things with their mid servers if you have all capability you should be fine but I went ahead and added this just so you can see it so you can have a capability of just Cloud management especially if it's a mid server that's own it's in AWS and all it's going to be doing is discover AWS things that's good that would make sense that you just give it the cloud management capability but if you don't have all Cloud management is the capability that you need to give your mint server in order for it to have all the patterns and things making apis on it that it needs to run to discover everything so okay back to the the discovery I usually just set mine um this part of it it doesn't matter it's up to you I because I'm in here on my own I just usually do once kind of do an on-demand type of thing um set it to active pick your start time but it's just like any other schedule we could schedule it however you want to do it daily most clients do it once a day or you can do it weekly especially if you set up the next thing in my series is uh it'll Mark changes and create it you can see things being created and see things being changed and it'll automatically call the instance through API and you can add things later so once you set that up this is less important unless you just kind of want to run that server so often but yeah so just come up with every schedule you want I'm not going to finish and run this because it's just going to create something I don't need in here I'll just go directly to my schedules very schedules and there you go so this is the one I created before the one below that gets created automatically so you don't have to create this one when this one runs I think for the first time it creates that second schedule as a child and we'll go run that one so here it is I will I've run this it looks like three times here in this instance but I'm gonna go ahead and run it Discovery now let's see how long it takes and while it's running I'm gonna go show you it works I'm going to go make another server in my system and show you that it's basically what Emily did actually I'm going to make a I'm going to make an RDS instance RDS stands for relational database maybe I'll have it online other one so I'll create another one I do want to wait till that first one is done now I don't want to get too much into this if we want to like create things in here just for testing purposes I have found that it's better to go with easy create you don't have to be a database junkie to know some of these things the cheapest one to go with is my sequel It's do that and then do free tier I think all of them have free tier but if you make the mistake and you don't turn the thing off these others cost more money than the MySQL one does so I tend to like go with that one because I feel like it's just cheaper a lot of this other stuff is if you want to log into the database build the database you need to you know set up Master passwords and things like that I just want to discover the thing so I really don't care about that part of it the name also share the windows DB that matters because that's what we'll show up in cnbbs so I'll go create this guy there we go it'll take a minute a few minutes to create that so while that's creating we're going to run Discovery again after that and show you that it pulled that in I did want to do that I wanted to demo that because you know all this stuff was already in there and because I've run Discovery before so I just want to show that you know it picks up another another item whenever you run it so any questions you want to break down on patterns and what they're running okay great that's what we'll do because we're going to see a new device being discovered and I'll go through that whenever we run this next so and that's another reason why I want to do something other than a ec2 instance the discovery picks up on you know which pattern based on what it sees out there is resources but it does start off as identifying everything in Cloud gotta figure out where I'm at too many tabs and too many too many windows so clown I think it's Cloud resources is what everything is first found under yeah each each time has that so I'm going to go AWS Cloud resource then you have your resource stock in class so familiar with cmdb classes are everything so yeah this is the roles and the policies and those things that I set up you know that was going through earlier to do some of those things that's those I thought everything was Cloud resource but apparently I'm wrong the I don't know if you go to AWS that's where everything is going to go I'll go to instances so you can see you know there's my Linux man my demo me edit knows that my Linux mate is off this one is on gives you all the some of the things with that and it falls under virtual machine instance class so let me look at see I'll go through the patterns and kind of show it doing that when I wrote do the discovery again where it was over here this thing doesn't finish quickly I'll just go into the old older Discovery statuses and start showing so here's all the stuff that it did whenever it ran I'm going to try to find that's it finding the discovery I want to find one of the servers I think I know what's going on I'm looking for the mid servers and mid servers don't show up as a device so if I had a device out there that wasn't a mid Server Like An ec2 it wasn't a mid server it would show show up I don't think I've seen my mid servers show up in here but my database should so let me go run Discovery again let me double check see if that's running it's there it should pull in even though it's starting so here it is running all the patterns it's going to get a list of all the the resources and patterns and it's going to fire off all these patterns for everything that it sees out there I know that the resources that I have out there I have ec2 out there and I have definitely a relational database so the relational database should be calling yeah DB cluster I think it starts with DB cluster and then when I data files what type of database it gets more granular with which pattern fires after that and we can see that in the discovery log when it's done you can follow the you know the pattern I think it's done with that so let's refresh and see if it's done anything and it's almost done CS service not being used there we go let's check this pattern if you go through all these steps you can kind of see you know what it's doing about what it's it's got pretty good description around it's doing I don't know what else you wanted to know about how they work but you know they kind of get they see a server and then they go break it down kind of like a typical Discovery Suzanne probe goes and figures out what what it is and then fires off other probes or patterns based on what it sees okay it's complete I just want to go show that uh it found the cloud database stwdb there it is and it's even showing the state that I see well it wasn't modifying earlier before it refreshed so the next session is going to be me going over here and changing something on the database turning it off changing the name whatever and then that coming over here automatically and updating the cnbb real time well close to real time so what it really does is it does a targeted Discovery when it sees a change happen it just goes oh well I'm going to go discover that specific resource and that's really what the system is doing so I'll cover that in the next year as well is how to set that up thanks for everyone for showing up [Music] foreign [Music]
https://www.youtube.com/watch?v=Rdn9DypfTTc