foreign Scott Murphy I've started the this series two weeks ago where we're talking about AWS Discovery and at the very first session just to recap we showed how to set up mid servers inside of AWS and then after we did that we configured the service now to discover devices inside of that AWS environment and then in part two we showed how to set up AWS config SNS that's simple notification service we set those things up in AWS to send notifications or events to servicenow and then we showed how servicenow can take those events and update the cmdb you know real time almost you know pretty real time so that this it doesn't have to run a discovery schedule to get updates from cmdb items so it'll do configuration item changes and things like that I've also since then created a knowledge article that has all of the documents I've been sharing I didn't put the Powerpoints on there yet but I'll do that to that same knowledge article and I'll get all those that knowledge article related to all the share the wills that we have been doing with all that being said I'll get started out on part number three where we talk about how to discover and BK AWS eks kubernetes systems but first I'm going to teach you a little bit about it because it's not a very well-known thing so I'm going to go through a little bit about what kubernetes is first so we can learn at a high level what it is just and I had to do the same thing I had to go what is this because they're asking me to discover it what do I do I don't even know what it is so it's a very complicated thing I'm going to try to explain it at a high level and I'll give you the resources to go look further into it if you want to so first what is kubernetes then we'll look into Caribbean kubernetes cluster and nodes in AWS it has to be there for you to discover it so that's why we got to learn a little bit about it create one and then we'll configure service now to discover the cluster and the nodes that I create what is kubernetes it is a portable extensible open source platform for managing containerized workloads and services makes a lot of sense right that facilitates both declarative configuration and automation it has a large rapidly growing ecosystem read more about it here so this link it's just a link to the kubernetes website in their overview they even have a Visio diagram explaining what you're probably traditionally used to how apps were installed on like your computer this is your traditional you install multiple apps on an operating system on a piece of Hardware your laptop blah blah that's simple right we all are used to this then if you know much about VMS you know you have a virtual machine on a big system multiple virtual machines supported on the same Hardware this is a little different this is kubernetes is based on containers it's the stir that thing over here where you create basically I compare it I'm a VM guy I saw I can think of it as an image it's a more of a configuration for an app but it runs it doesn't run on specific Hardware per se it can it can be requested to run on multiple things of Hardware based on load so if the load gets really high on these on the container runtime if it maxes out it can spawn another node you know real time so it adjusts itself and goes lower it uses more or less resources based on load which is awesome especially in this digital world or virtual machines and you're using AWS or Google cloud or something like that and you get charged based on the resources that you're using whenever you put things in these containers if you're not getting a high load then why do you have 10 servers running a website that no one visits in the middle of the night so if you have a high load during the day they spawn up they build containers containers multiply they get bigger and you have more instances of it running so that's the highest level of kubernetes that I can come up with and translate it to what you kind of know of today so I'm going to go back to the PowerPoint and we'll go yeah there it is and yeah this is the same video I just talked about from their website but there is a lot more to look at that website if you want to look at the concepts and things like that and touch on and read up a little bit more than what I just showed you but I think that concept is good enough to grasp what we're trying to do here that's what kubernetes is and what we're trying to configure so it doesn't come naturally service now you have to do a couple tweaks to it to get it to work and also in kubernetes you have to give permissions and roles and I'll show you that now so yeah demo time let's build it so the first thing I'm going to do is show you where that is it's called elastic kubernetes service that is inside of AWS and I currently do not have any clusters in the I think yeah USA's two data center So currently don't have it in there you can do it like this where you create but if you do it this way it takes in my opinion longer slash there's a lot more steps to it I'm going to do it with commands command prompt commands and I'm going to use the cloud shell that's inside of AWS to do that so go to my list I do have a list of all the commands I've tried and it's crazy don't pay attention to that let me find I'm going to get this into a document I haven't done that yet but that will also be part of this I do have a Word document on this but it's assuming you know the cluster is already there and things like that I made it for the client they already had that stuff so I just had to show them how to use it but for my purposes for your purposes if you decide to go create this test it out on your own play with it get used to it you'll need to know how to make a cluster and you can go do it the graphical way sure I'm not going to show that because this is honestly faster so it is a little bit more technical because you're using you know command prompt or command line but it is faster to do it this way so first thing I'm doing right now is I'm installing I'm going out and installing something called eks CTL that's what I'm using it's a library of commands that allow you to build clusters easily so let me go ahead and get that installed at my cloud shell and you'll have to run this each time you open Cloud shelf you close Cloud shell you have to reinstall eks CTO okay so it's that quick and we'll go back up to my commands and this is me going to create a cluster right here explain the command as it's running so I'm going to hit enter it's going to take a while to do this it takes I don't know 10 minutes maybe so I'll talk through this as that's going on that eks CTL what I just installed is the command I'm now using and I'm just doing a create cluster I'm giving it a name this is the name that I've created with it you tell it the version of kubernetes that you wanted to use I'm probably not use this is from December of last year so there's probably a newer version now but then you declare the region you know I'm in the US East 2 right now in AWS it's going to show up there when I'm done whenever this is done you give it a node group name this can just name here which is I have it set to Linux nodes that's what that's going to be then you declare the node type T2 micro that's always the safest node type to pick for anything even when I were we were creating the mid servers we picked all of this stuff graphically when we're making ec2 instances and that's eventually what these nodes turn into whenever the the cluster gets created the nodes the Pod all those things I'll show you when this is all done they turn in they'll look like ec2 instances they'll look like servers that are in there but there you go so yeah and then you declare how many nodes at the very end how many that you want there's probably a lot more other commands you can put on here but this is the bare minimum I needed to get this going so if you ever want to look more into this it's eks CTL um you can Google that to see more information on how that works and what that is and AWS is a great resource for this stuff if you go to their website looking for documentation on anything that's where I found most of this stuff it was way too much for you for me to package together but it is very easy to find very easy to follow like all the servicenow documentation that we have right uh I will take this time while this is running to kind of show you the path I took to get here I think I start off my crowd Discovery document with that yep this is the community article that I used to go with this document isn't a knowledge article that I've already published for the share the wealth AWS discoveries but it gives you step by step you know how they configure this with screenshots blah blah blah which was simple enough to understand to take me a while to get through some of this but I thought a video would be great I haven't seen a video out there for any of this so that's why we're doing share the wealths for a lot of this stuff because there's no videos I like seeing those so that's why I'm providing the video on how to do these things but this is was my original source of how to do all of this but it didn't have the things I'm showing just showed now how do I create a cluster how do I do all that wasn't in there so I feel like that's good context to have so that you can actually go implement this stuff otherwise you're waiting on a client to have it and if it's your first time what do you do you just tell them go do this no I like to have a little bit better approach that I like to have done it already if I'm going to go tell them how to do something that way I'm not sitting on the Cog going well I don't know why that didn't work or did you check this I can have some kind of context to go did you check this did you check that this is how I did it so you can see how it worked you know typical things you all know what I'm talking about world Consultants here so you want to I like to have done it already so that's where I started out at still waiting for this to create okay at the time this was written you had to download this update from service now and install it it basically just creates a script to include or modifies a script include that whenever the discovery schedule that we you know we'd set up prior whenever that runs and it identifies the kubernetes cluster inside of your AWS environment it will then create another Discovery schedule just for that cluster and I'll show that when we run through this I don't think I've ever run through this on my current um PDI I think it stays on all the time for me because I have AWS connected and Discovery running at a good regular rate so hopefully it should be alive we won't have to wait for that too okay good we're good to go so back to my other browser so that's still running let me go take a look at that to see if it showed up yet yeah so you can see now that that cluster has popped up in there it's creating doing all those things in the background to me that's just so much easier kicking that little command off and it going through and creating the cluster the Pod there's like three or four layers of things you have to go do so you can go through it that way if you want if you want to go through the graphical way but even um you know what I've seen with other clients that I've seen work with them that's how they do it as well they do it with command line only they really don't go in and do it graphically so this might be a good time to jump over here do the search Now set up and then we'll go back we'll do a few more configuration items in AWS to make sure we have permissions so that when we run the discovery it works so this update set I do have downloaded I don't know that I put it in the instance so let's go look at that we just got that started up scary schedule API updates yep that's it okay okay I've done a lot we've downloaded that we've installed that uh this again this is our descript include that works with the kubernetes pattern to create Discovery schedules for the specific kubernetes clusters that it finds during Cloud discovery then you're going to need to install AWS CLI on the mid servers that are doing the Cloud Discovery here's a link to that download [Music] I think I already have that on there but I may not I'll have to check that so that means I have to go log into my Discovery server and see where is it is it in North Virginia let's see yep so I don't know if I've showed this before but that is how I connect to them I just use RDP I don't know what passwords I'll probably get that oh I don't know my private key Either the latest one would probably be that one hey that was right so I've got that saved you know I could probably go in here and do this so I'm trying to again I'm trying to make sure I have AWS CLI installed no I think it would be in here or something else okay so it looks like I need to go there so in order to do that I have to go download that unless I just double click that maybe I have already installed this well I'll show it anyway I must have double clicked it yeah I downloaded twice today so it's not installed but I think it's just a vanilla install next next next go go and there's no configuration after that it just has to be there I'm guessing it uses the pattern must use some files on here um to do the discovery properly that's about all I know it was in the document of how to do it so let's see where did it go like to see hey I completed or something now the installer's still running down here yeah okay there it is and I know by now that the cluster is created we'll go take a look at that next okay so we have that installed we don't need to be on here anymore so that is the only step where you have to have that on your mid server now I know in the first video we set up a mid server in Linux uh Emily how do I took care of that how to do what I just did in Linux I haven't done that yet so I really can't show you or tell you but the website's still all the same I'm sure you kind of do the same thing where you you give it a URL do a w get and get the installer package for Linux something like that is what you'll have to do so okay I'm done with this session let me log out of here I'm pairing up my document okay there's some properties we need to set and things like that so let me go back and make sure that my cluster is here I'll show I'm still waiting okay so it's got some progress It's Made the cluster now it's working on the nodes okay so it's still taking its time that we did that first yeah let's go back to here and I think I can show you what the cluster looks like now I switched data centers so I can't there we go so the cluster is active just to give you kind of a I'll explain some of these things but uh here it is um here's an overview of it here's the resources the pods odd templates jobs crime jobs all kinds of things that are in there some some things on the cluster the workloads so once those nodes get created they should show up in here yeah they should show up in this spot so they are ready so they just must have just created so there they are so you see them here is Linux knows inside of this cluster they should now also show up inside of ec2 as well whenever they're up and running boom boom so we have that name that I created the cluster name then with the Linux nodes and then node one and two so they don't say no wouldn't you I can change it if I wanted to I'll go back a screen to my ec2 dashboard you can see the instances running my IPS volumes things like that or the auto scaling group is what I was talking about earlier that's key these things Auto scale to need they go down they go up so that's a really cool part about these the kubernetes things clusters okay so back to the document let's get uh servicenow ready to go uh to enable the eks creation we go to these sys properties list over there make it kind of easy for me I'm going to type it in and misspell assist properties two three times I should have to find my window there we go okay this is properties and what were they scni Tom pattern K8 s there we go okay so what am i setting the midservice or the name of the mid server which was set up above and create schedule enabled to true okay we'll be in that app yeah and it's every name I'll create a discovery schedule which targets the AWS account that contains the Clusters we've already done that sure you have okay this is important the mid-security policy uh that will allow connections to amazon.us.com so I don't remember where those are all right that's a menu upgrade okay new active yep okay without that it will fail to connect whenever it's doing the kubernetes discovery okay back over to AWS now servicenow should be set up we just have to execute the discovery schedule after we get all these these things done the mid server ec2 instance must have permission to read AKs objects so we're going to go to the permissions and and here I say there are many ways to do this I insured my mid server has a role with a trust relationship to eks and eks ECS Services um also view only and or read-only access permissions I think in my previous videos you saw some of these things because I've already put them in here I don't think that I have to go do it again but I'm going to go check just to make sure so let's go back to AWS and we'll go into I am identity access and management or at any nxs management we will look at our roles and the role for the discovery server is servicenow ec2 role in my trust relationships yep that is already there so ec2 is there that had to be there for the discovery that we did before the initial Discovery this is to allow eks so that trust relationship was added also um it's showing the view only or read-only access in the permissions so I think I showed that just by uploading it and I have more things so I must have been doing some other things too but read only view only they're there I think you only need one of them but playing around with it all done more than I needed to so next we need to in each eks cluster in the AWS accounts you must have the following and I am mapping between the mid-server role which is represented by this rnid and then it's going to be to a kubernetes username so that's something inside of kubernetes that we're going to go configure a little bit later after this step but so let's go run that so that command we're gonna go back to the command shell hopefully it's done with everything it's doing by now perfect it is back to my trusty little sheet more lines to run so there's that command that I just showed um have it already pre-configured for this cluster name where it's at the r and ID for the servicenow ec2 role so ac2 rule RN is this right here so that that's important for building that command okay back to here so there we go so there's the rule there's this the username service now discovery that I'm going to set up inside of um the eks cluster soon okay so that can create that created that entry next I have to go into the cluster itself so this command is an AWS command to update the cube config the kubernetes config for the cluster that I just created so now I should be able to run that in here looks good added new context okay so that's what we're looking at down here at the bottom screen now I can talk to that context and this is the next thing I'm going to do add a cluster role which allows a reading of all objects in the cluster so there's two of them there's one there one here what does that look like in how I input that in the system that article that I read that's verbat on what it had I was like well how do you do that this is what you do you run this command here and yes paste it in it puts that file in memory so you can access it at a later step I'm going to this is one of the files that you just saw in the instructions there's two this doesn't do anything per se it just gets the file in in the Shell program so that you can apply it click on that and that okay now we're going to apply those configs this is applying that configuration to the cluster now much about them other than now it's a type of configuration type file AWS is that's how they run everything is up again well basically but any command line things configuration files is yaml yamliable okay so those things things got created what does that mean it it's a way to command line something that I don't know I don't think you could do it any other way I didn't find out how to do it in here but it's ins now that I did that those things show up in my cluster config so when I come into here I go to my resources um gotta figure out my user shows up in here somewhere here's the config map that I did where I mapped the amount it's in here looking for that service now Discovery user oh I may have I just don't have it there we go hit the roll binding that's what I just did okay so they should be oh add a filter on okay that's gone so the role binding should have servicenow Discovery in it sorry I'm just remembering I know it's in it what we did in command line added something in here I don't have it off the top of my head where it's at anyways the intention of what I just did was to allow the mid-server read access into the cluster and I'm pretty sure I'm just not I can't see it right off top of my head it's in here allowing that that's that's the purpose of the commands we just ran we're ready to go uh do the discovery so I'm going to go ahead and do that and if I have time left over I'll go look back into there and find in the cluster what I was just doing it is in there it has to be I just ran it so back to servicenow okay so I'm going to go to Discovery schedules and right now we just have that master account and schedule we've never tried to run a kubernetes discovery but it'll it should next time we run this discovery when it finds that kubernetes resource cluster it'll spawn off another Discovery schedule for that kubernetes to go do it that's running this shouldn't take too long this should take only a couple minutes what if we had more than one mid servers for this purpose good question I would I haven't seen it used by more than one more than one mid-server if you do a mid server cluster if you wanted to to do more than one I would recommend that so if you're declaring in the configuration that mid you can just put the cluster name in there that way you've got more resources to go do the discovery so I don't know I haven't done that personally right I don't want that going on in my test environments I don't want to have a lot of machines in there and cost more money but uh that would be what I think you could do make a mid-server cluster and and then whenever you're going to pay for configuring to do this um you can definitely just put the cluster name in there instead of the specific mid server and when I get done kind of showing you all this there is um the ability to put a mid server in one of these containers you can make a container you can put a mid server inside of one of these so that it can scale and go up and down haven't done it yet I don't know that anyone's done it I think someone here has done it in a company but I've never done that that's my next goal um along this path just a challenge for myself I haven't I haven't had a client need it but they did ask about it and I pointed them to the when you go to the mid server installation website it's there as a Docker container Docker is another word for Content another company for containers it's like kubernetes again I'm not an expert on that type of thing but it a Docker image Docker container is what you can use to make these kubernetes clusters notes okay so Discovery is complete let's see what it found you can see in the discovery log um a lot of times it found something so I'm going to type in kubernetes or kuber didn't find it okay maybe I'm looking well I'll look for that later let's go see if it made another oops because it was it did not should have made a schedule here if it worked okay so I do need to go look at that discoveries test see what happens wrong so it did run cluster oh that's a roll okay there we go can't find any cluster in this service account pattern name AWS kubernetes co-creator okay so it didn't see the cluster so it's saying because it doesn't have permissions okay I do this is coming back to me now I remember whenever you do this or organizations so like I explained previous ones if you have a parent account like I'm I'm running this with that master account my cluster is in that secondary account of mine and I must not have that trust relationship set up properly but I know what to do so let's go look at that so it's all in I am it's probably why I set up all of that those other roles and policies and things that I've seen somebody go check that out let me look at my syndrome [Music] it may be something I have to do over in master account let me check my master account permissions [Music] that's the risk of setting all this up for the I try to take everything down and then build it back up as you can see it that's the way I like to show how to do things if it was already built there's not much to show so sorry about that but I'll get uh hopefully get this fixed right here and now okay yes good stairs well foreign my user in here let me look at my commands I may have yeah no that's in there servicenow slash discovery and that's what I gave my username for my cluster this is where all that is that should give me the rights to do what I needed to do this is these are my attempts from the very beginning to just log everything I ever ran in Cloud shell it's a bit unorganized and that other thing I'm showing is a little bit more organized than that but there are some cleanups after you're done to do the delete cluster for whatever the name of it is which is nice too it's one stop thing run it you're done and you quit paying for this thing this thing to have this node running like this is like a dollar a day maybe three dollar can't remember but it would cost you a little bit of money to have this thing running so I like to destroy it once I'm showing it which is why you know have to reset everything up again every time okay I think I've ran the commands properly but I'm not seeing them show up right oh maybe it was my Cube config and I just set that one up right yeah the right name there hmm no check my clown shell make sure nothing failed I didn't see anything about when I ran them and it goes to the config map foreign resources so whenever you do a resource you can restrict which resources they have access to and I think that command that I put in here didn't have star in it my client asked me if they could restrict the things that it can see I was like yeah sure you can just name the resources by name and this command here but the original article has a star right here so I'm going to re-run this with a star in it that could be it maybe I've done that incorrectly so maybe if I follow my document that'd be okay yeah so Resources with a star get watch list okay let's do that again so maybe that's it that's all I found it looks wrong to me so far sure that was the cluster role okay I'm going to re-review the other one that's the service notice curvy user that should now be a cluster in the cluster role binding pretty sure that's the same was what I entered earlier to this file yeah that looks fine just reapply it doesn't change so I didn't do anything okay I'm gonna go rerun the discovery and then keep looking this is my discovery global role binding okay there it is there's my user servicenow Discovery with okay that's good so maybe it was just the resources that I didn't have a star there and that hopefully was it so we will see that this is what I was looking for earlier all of that was in the file if you looked at it line by line the role binding looking at right now is uh there's a read-only cluster rule that was created with all of these things and then there the role binding is what I'm looking at what I've just had on the screen what's called Discovery Global and then the user is this and it ties it to the cluster role from before called read only so that's what allows it to talk to the mid server reply back so I think I had that off a little bit hopefully and that was it so yeah I'll run through that again here so if you go to overview resources the resources inside your cluster you go to authentication authorization you can see the cluster role what I created was read only there it is 388 minutes ago and that roll dice to this this here we go okay so it's not if it doesn't work this time it's not because of this it's probably some of the other rules that I have with uh and trust with the role that I have created to service now just go re-roll it's completed let me see what the kubernetes did this time can't find any cluster on this service account um got less than five minutes left a pressure is on to make sure and show you this thing works um if I can't I'll um definitely you know put something out there added them to share the wealth or something like that to show what it was that got it to work apologize for not being able to show that but that this kind of just shows how complicated this stuff is right um I know I ran into the same situation before and I did it the first time oh sorry I couldn't do it during the time that I needed to be done to show how it works uh but this is the the gist of it this is the complexity of it yeah I'll take a couple more minutes to try to figure this out but if I can't sorry it'll have to be shown later but I will get that out there see I feel like I did uh administrator access I feel like I did some I cheated and gave it uh the role all access she shouldn't do but uh it could be a way to get around and show it working I do have it don't I I do I did give it administrator access already so that should work maybe we'll check the other account okay let me give it that policy here one more try so what I'm doing I did I had the administrator access on the master account side to the same role that exists on both um accounts so I'll see if that works they shouldn't be adding admin access that's not the answer but that's the shortcut I'm going to try to take to show it I'll figure out the true reason why and the the whole thing the complexity of this is um why it's not working is I'm not doing any I could set up a targeted Discovery just on my sub account but that's not in a real world Solution that's not a solution that they want um when I pitch this to my client they said you know we don't want to have to set this up a schedule up for every account that they we have we have hundreds of them so I'll say oh okay um so we worked together we figured this out by um doing the master account Discovery you can assume role in any account it can find the Clusters there so we did get that working apparently I don't know maybe I'm missing a step but I thought I went and updated the documentation for whatever we did to make it work I'll go review all of that and make sure I didn't miss anything okay so what do we have couldn't find any clusters okay and so I'm guessing that's running on this server yeah it's looking at Cloud account this cloud account okay so looking at that cloud account I think I know what to try so when I set up my role identity mapping I use this number may have to use my rnid for my other account maybe I grab the wrong one out of here so that's the number I need okay so I'm mapping it to the master account service now ec2 rule which may be where it's coming from because I'm discovering from the master account assuming rolling the others that's my thought process I don't know exactly that could be one maybe that's the command I ran instead so I'm also going to go do the mappings again but with this other dimension now it just say mentions the names doesn't have that in there okay I'm gonna rerun Discovery again to history [Music] got that there so it's only looking on the master account it's not looking on that sub account so it looks like to me okay feel free to drop everyone I'm gonna go ahead and close the call I'll look into this and uh I don't know how I communicated I'll ask Brett what do I do maybe I have a session for where I show why this didn't work [Music] foreign [Music]