before we get started remember that we have additional resources available for you now you can find our Ai and intelligence Community Forum it's available 24 7 for you with articles and content and this is also the place to go if you want to ask questions about our AI products to go there is one simple link it's ascend.works AI if you are watching that on YouTube remember that you can subscribe to the servicenow now Community Channel and get all the latest updates uh before we get started please take a minute to review our Safe Harbor statement and remember to check with your account team whenever you're making a purchase decision to make sure what's available at the time of purchase in the product if you ready so for you it's fresh ideas it's a great content to better understand and have practical guidance on our product the session is recorded and then shared on our YouTube channel if you have friends or colleagues who couldn't attend today feel free to send them the link to the recording but since you are here today you can Leverage The Q a feature to ask all your questions during the session today and with that I'm gonna hand it over to Sean to get started terrific thank you Eric uh good morning good afternoon good evening wherever you are so my name is Sean Hughes I'm AI ecosystem director at servicenow um and I'll be stepping you through this morning's a session on introduce production to servicenow research joining me is my colleague Bruce teslakian do you staff research scientist at serve now research and with that lyric thank you if you want to get us started great so I'll spend just under five minutes going through some background setting a theme for you and then we'll move over to peruse where we'll dig into scientific AI research at servicenow followed by a brief q a so um if you're a servicenow customer you've likely seen this message already the world works with servicenow and that no matter where world work happens Digital workflows Drive productivity and great experiences you're also probably aware that servicenow AI Pros are intelligent workloads that drive our great experiences these include onboarding new employees and also processing new customer loan requests for example at a bank but aware will work happen in the next five years how can AI be leveraged to optimize and enhance digital workflows whole productivity be measured in the near future of work and how will AI tired automation change our experiences and decision making well this is where fundamental AR research comes in and it's one of the ways that we can try and answer these questions but with that I'd like to hand over to peruse as I mentioned before peruz is a research scientist and the research lead of our low data learning program at servicenow research Surfers please take it away thank you Sean for the introduction and thank you everyone for being here uh it's my pleasure to be here today and tell you more about the work we do at servicenow research so what I will do in the next 10-15 minutes is uh give you an overview of the type of things we work on the different Focus areas that we have when we're doing research and um and uh maybe tell you a few uh research topic examples uh or papers that we've done in the past what you'll notice here is that our research is mostly carried collaboratively we focus a lot on having strong collaborations both internally at servicenow and also also um across Academia with external partners and we all we publish our work in open scientific journals and Publications and this is a part of our core values as researchers in servicenow because in addition to um making sure that our work is reproducible and understandable by other researchers we also want our customers uh to understand our work to um to have transparency about how we're doing things uh in the research space uh so so what do we do at servicenow what we focus on in service now is making workflows be smarter and the way we want to do that is you want we want to integrate AI so that uh these workflows are also more efficient um and we want to do all that in a very socially responsible way at the same time providing transparency and um and respecting um all uh all values that are out there um we also support the global development of knowledge in AI globally through our various Partnerships and open source projects that I will give you an overview of but first let's talk about who the people are behind the research that takes uh place at uh service now uh I will start with our research advisors our research advisors are external uh through servicenow who are advise us as researchers so at the Forefront we have Professor Joshua Benjo who is um I think the second most cited computer scientist in the world he is the winner of the Turing award uh which is um which is like a Nobel Prize for computer science uh for his work in AI he is a professor at the University of Montreal and uh is a founder of the Mila lab which focuses on machine learning and AI uh next we have Professor Chris Manning as a research advisor for the human machine interaction through language program which I will mention in a bit he's a professor at Stanford and uh is a very decorated researcher in the area of AI uh and we have also a professor Stefano Emerald Airman who is also a professor at Stanford and is a consultant or an advisor for us uh at service now research next we have our uh primary fundamental researchers which conduct research in Three core areas of research um in the first area uh that we call programs internally so the first program is about human machine interaction through language this is the group who focuses on um language models or modeling language through AI and so anything about text or language or chatting with machines this is the group that focuses on that uh that kind of problem next we have the group about human decision support what this means is that the research Happening Here focuses on helping the user make better decisions and we do that by focusing on um Concepts such as causal reasoning uh giving causality uh mindset to uh machines to AI trying to get AI to have a numeracy meaning that understand numbers understand how time series work uh be able to predict and forecast uh time series uh values such as you know you have a stock value that changes over time and you want to be able to say something about that in the future in a future point in time so these are the kind of problems we focus on in this group and finally we have the low data learning team in this program what we do is we try to find ways to train models machine learning models with as little data as possible now as machine learning models get bigger and bigger and training them becomes very expensive very time consuming and these models are very data hungry as in you will need a lot of data in general to train them and adjust them to your needs but there are some things you can do to make that data need smaller in order to reduce your costs and time and this is the these are the problem that this group focuses on like how to do something smart in order not to need so much data to train these models to adjust these models okay so we also have aside from the research programs we also have uh research labs now these are um formed on a needs basis so as soon as we have a let's say in the research group a technology or a concept that has the potential of transforming into a product a very wide product or which is um more applied we spin off a lab and that lab starts looking into this in a more concrete way now the two Labs that we have right now are called AI trust and governance lab and the second one is the emerging Technologies lab so the AI trust and governance inspects and develops all the methods in which we can ensure that the machine learning models we are developing are trustworthy they are transparent they uh do not have that bias or they minimize the bias that they are safe to use and they conform with all the regulations and governance laws that are out there and this is very important aspect of AI that uh many tend to ignore but is at the core of safe Ai and we have a whole Lab dedicated to this uh concept the second one is emerging Technologies lab this is a lab that specializes in in different Industries and tries to export the research that we do in the research group to these industries like banking or manufacturing and what are the specific things products that they need and how do we move the research happening in the fundamental research group to adjust it and move it to applied products in these industries and of course we have our research management who uh undertakes the task of managing the operations of all these research groups and the researchers and the interns that we have and the programmers and uh and the infrastructure that we have to run so uh now let me tell you a bit about um the theme about these groups um so for every group or every program uh we can summarize it as the question that this program is working on so for the human machine interaction through language the question we are asking ourselves is how do we enable servicenow users to confidently make the right decision at the right time um so actually this is not this is the human decision support program that the text is mixed up there so for human decision support what you do is we try to enable service style users to make decisions uh in the correct way uh for the human machine interaction through language group what we want to do is to enable the next generation of uh language interfaces human and machine language interfaces for the little data learning group the question is how do we efficiently train AI models using just a few labeled samples for trusted governance we want to be risk AI for servicenow and for our customers while also adopting uh or supporting adoption of AI at a very large scale and finally emerging Technologies the question here is how can we demonstrate ai's ability to accelerate these digital transformations we also have many research Partnerships uh together with other uh other Enterprises or mostly academic partners and here our goal is to support the global development of knowledge in the AI space and we also want to make sure that our contributions uh to service now are rooted in the most Innovative approaches applicable to Enterprise AI which is what we focus on so to mention some of the projects or the papers that we have put out there um if we can move those slide yes um so this is a grid of uh different uh different specific uh projects or papers or research Works uh that we've done over the past few years grouped in a theme let me touch upon some of them some so one team deals with model training and this is about figuring out ways to efficiently train our models to train them quickly needing less gpus needing less data Etc so there's a class of papers that we have in this space we have a whole area of summarization Tech summarization whether you're summarizing a long chat or whether you're summarizing legal documents or very huge uh huge pieces of text this is another area knowledge synthesis is an area that deals with um trying to either complete the missing uh information in your databases let's say if you have a piece of database it doesn't have to be language and it also deals with the question of um how do I interface my database uh my Enterprise database let's say with a language model that needs to have some concrete information dig out some concrete information and uh in order to answer a question text to code is another area where uh we would like to convert natural language into a piece of code that uh in order to help decoder or the user uh develop its uh uh it's a small program intent classification with gpt3 is another project that was done here and this is uh trying to classify that intent of the user uh through that natural language again um reasoning is a whole big area by reasoning we mean enabling machine learning models to reason like humans and um currently many of the language models that we see out there lack this kind of reasoning capability they look like they understand but they really they don't and um and there are many techniques that we can develop and integrate into these language models so that they can understand Concepts like causality if you do this it's going to cause something to happen and this kind of links and understandings is uh something that uh models need to have in order for them to be safe to be reliable uh and uh and uh to be usable in an Enterprise context uh there are other uh we have a lot of work in computer vision in generative AI uh in generating uh you know images from text or other kinds of inputs uh we have uh work in continual learning which is about to have a model that is trained on certain data how do you keep it updated how does this model continuously learn as you interact with it and this is not a very simple question and there are many ways of addressing it and we do research in this as well uh we have applied our research to other settings such as climate files where we have some work uh that um that computes emissions uh with machine learning approaches um that uh that tracks climate uh change very uh through different means satellite images uh Etc again all of this uh using AI uh I will dig into some of these uh projects in a bit more uh detail but before I go uh to that let me quickly mention uh about uh the the research impact that we uh we have and how to measure it so the research we do in the research group in the fundamental research group is um is very Cutting Edge um not all research uh projects make it to publication must do but uh not all and even fewer make it into a product so it is usually a very long process between you have an idea that you want to explore and this idea is already in a product there's a very long uh maybe six to month up to two years it could take because what you're doing is very risky kind of investigations they may or may not work but if they work it's going to be great right and we constantly push for that but now we publish our work uh everything we do is uh you can find on Google Scholar where uh if you search for a paper you will find it and find all the details pertaining to it we publish code uh the code we use to train our models uh that are in the paper and um one way to find out how impactful published paper is is uh through a number of citations how many people how many other researchers cited this paper and this is what this slide is showing that in Google Scholar it's very easy there's an easy way to find out how important let's say or popular this paper is and as you can see one of our papers has 27 000 public citations which is a very big number given that given that it's not a two old paper and our research has many of our works has a lot of impact and in the research community in the academic community and a lot of that also translate or will translate into our products that we provide to our clients and if we go to the next slide this is um just an overview of the most important papers we've shown uh during the year and uh and you can take a look at it if you'd like yeah okay so let me just quickly give some spotlights some uh projects uh so um I have selected six featured research projects uh here um and we have many many more but uh let me just go through uh some of these uh the first one is fashiongen uh it's uh one of my favorites personally fashion Jam was a project we did a few years ago which was about uh generating images from text so we had a uh we collaborated with a company that uh that sells fashion items like clothing high-end clothing uh and they provided us with a data set of pictures of these clothing with text descriptions of the clothing and we designed a model that uh given a text a description of a piece of clothes it would generate the picture that your description and that matches your description and we had a whole competition around it and many people participated in this project and we also did other things with this Pro with this data set like recommending pieces of clothing that you should buy that match with what you already have based on your wardrobe and this was a recommender system uh and this was one of the projects uh that had a lot of impact at the time uh we have a project that was recently published actually called tactus this is the time series related project uh this means that this is a project whereby if you give this model a huge number of Time series let's say a bunch of stock stocks with their value over the past three months it will be able to forecast uh the the value of one of these stocks in uh at a future point in time and this model had a lot of accuracy it's a very huge model which has the potential of being transformed into a foundational time series model if you'd like that understands time series and how time series work and this is uh this was recently uh published at uh icml um now the next one is tapioca tapioca is a project about conversations about interacting with an AI interface it's um IT addresses a particular aspect of uh chatting with an AI interface which is about uh switching contexts when you chat with a machine and you have a certain theme or a topic you're talking about you sometimes would switch context and start talking about something else that's the natural way of conversation goes and uh language models sometimes are not good at that context switching and this was a project that was dealing with that aspect of uh chat Bots and uh had a model that would uh address that in a successful way Picard is a text to SQL project uh you know SQL is the language of databases uh where you um in order to query a database you usually need to know uh SQL language and Picard is a model that given a natural question natural language question would translate that to an SQL query sentence based on the database schema of your own database and this would enable users to uh to to query their database without having to actually know uh how to construct very complex SQL statements um Azimuth is a project uh that is for machine learning researchers uh it's a project that helps machine learning researchers or practitioners better understand their data sets better model predictions uh doing error analysis with a very nice visual interface this is an open source project it's up there for anyone to use and it does many things like systematic similarity analysis and sales c-maps and uh all the tools you need as a an AI practitioner to uh to to explore your data and finally uh we have a big code which is our biggest project yet uh in this space and uh it's the most recent one as well big code is uh a collaboration between servicenow and hugging Pace it was announced on in September uh last year a few months ago and um it was a collaboration among 500 more than 500 researchers from 30 countries uh and what we put forward is a language model that converts natural language to code this is a foundational model meaning that you can build on top of that your specific needs your uh train it to your specific domain um and uh the model itself is called santacoder uh it has one billion parameters and uh the important thing about this project is that it is done in a very transparent way in a responsible way uh instead of training on all everything that we could find online we asked users to um we had gave the users the option to opt out of getting their code being used as training data sets it's uh the model itself outperforms many of the large models out there and uh it is a very good start as a collaborative open responsible uh Foundation model uh which we believe should be the way that businesses should do uh modeling AI modeling within their companies um so I'm running out of time so let me just quickly say that um the future of AI uh I see it at very exciting it has the potential to affect us on all personal Enterprise and at the humanity level um for the personal level I would imagine having a cognitive body uh with me all the time giving me advice on various things that uh I would like to know about uh from the Enterprise automation perspective it has a lot of potential to automate a lot of The Works uh that enterprises uh need automation for and finally um its impact on drug Discovery will be huge in the future that's something I am personally very interested in and on the effect on how we um how we understand climate how we track climate change and uh and how maybe we can deal with it and make the correct decision at the right time to uh counter it so I'm going to hand it over to Sean for some parting thoughts thank you that was great um I hope everyone found that uh insightful and valuable and I know we covered a lot of ground uh in Cruiser session there's so much to talk about uh we do look forward to having follow-up sessions so do feel free to put into the chat the parts of the presentation you found most interesting uh perhaps if there was something that was thought provoking for you that you learned about today that you'd like to know more about let's see if we can try and address that for you we're going to encourage you to visit our servicenow research website and this is on servicenow.com forward slash research you can also follow us on LinkedIn and Twitter serves now research on LinkedIn and at servicenow rsr CH on Twitter and on Twitter you'll find probably the most current uh real-time news and uh get resp you know CR responses to the community everything is done very visibly uh just to point some notes on the big code project it is an open science collaboration it is active today anybody in the world is able to go and visit the big code project um and if they have a research interest if they have an interest in trust and governance for large language models and once it helps shape the future of of where we go with this uh the the collaboration is open so there is a a form where you can request to join um and anyone who does join really it is a research project so you know there is a little bit of screening up front just to make sure that that researchers are first and foremost the ones participating but we also do welcome practitioners and um even legal teams to to join our legal ethics governance working group within the project this way uh you know you can participate in the dialogue and what's more all of the models data sets model cards data cards uh code to reproduce results from experiments from Big code project they're all published in open source so um as was mentioned by peruz you know we've actually got the attention of the European parliament's Innovation lab and we're we're they've actually requested us to meet with them just to explain you know what we're doing because they they've been following the project and for them this is a model of how open science uh should be done so you know encourages to check that out and with that the Rick thank you very much for hosting us thank you just before we go we have a one quick question uh somebody was asking if there is a specific group uh amongst the group that we presented that is working on on for example the AI search feature uh would you say it's it's shared and collaborative effort that then leads to new products sure you know our fundamental research scientists they focus on the you know long-term five-year plus type research uh one of the teams that uh through spoke about was the emerging technology lab as well as the AI trust and governance lab so our emerging technology lab is mostly comprised of Applied research scientists whereas the other researchers are fundamental research scientists and the applied research scientists they take our research they package it up into proof of concept applications they they really dig into to try and fully understand it and then they have a very close working relationship with all of the the product teams that are developing servicenow capabilities like AI search and so you'll see that that all the research that was presented there's bits and pieces of it that apply to uh different modules different AI powered modules across servicenow so through our emerging technology lab we do work closely with the arch team there's no one particular person it's it really depends on what is the question what is the problem that the AI search team is getting as a big hard rock from our customer base and then is there anything from servicenow research that is potentially relevant to tackling that project yeah thank you thank you very much for the answer so thank you thank you for the both of us to presenting today uh and it was great to show the the benefit of having an open science open governance approach and how it's gonna increase the ri for our customers using the servicenow platform thanks Eric thank you thanks everyone