This video is an overview of the ServiceNow
NLU Workbench test sets management feature. To give users the best experience with Virtual
Agent and other NLU applications… …you’ll want to test your NLU models to see how well they’re predicting the correct intents from the users’ utterances. You’ll want to test your model when you’re building and training it… …and when you’re refining it when it’s in use. To help you test your model, NLU Workbench
helps you create and refine a test set... ... a list of test utterances with their expected intents for the model. NLU Workbench also helps you test your model using your test set… …and refine your test set over time as your model evolves. When you create your model… …a default test set for the model is created along with it,... ... ready to be populated with test utterances. You can create test utterances for your model in a few different ways. You can manually add individual utterances
and their expected intents. You can import test utterances from CSV files… …or other existing test sets. And when your model is in use with Virtual Agent… …you can add actual user utterances from
the Expert Feedback Loop. To test your model thoroughly, your test utterances
should be realistic,... ... things you’d actually get from users. And you should include at least 15 test utterances for each intent. The more test utterances you have and the more realistic they are,... ... the better your test set will be. You should also include some irrelevant utterances to make sure the model does not predict an intent when it shouldn’t. You’ll want to make sure there are test
utterances for all the intents in your model. NLU Workbench gives you a Test Coverage score,... ... the percentage of intents in your model that have test utterances. You should have at least 60% coverage before
you actually start testing your model. And ultimately you’ll want 100% coverage. If your NLU application is available in more
than one language,... ... you’ll have a separate model for each language. You’ll need a separate test set for each
model, in the same language as the model. The NLU Workbench Advanced Features app includes
the Expert Feedback Loop feature,... ... which captures actual user utterances and the resulting intents from Virtual Agent. You can review these utterances to verify the intents… …and add them to your test set to make sure
it’s testing the utterances you actually get from users. When you’ve trained your model… …and your test set is ready… …you can run a batch test in the “Test and publish your model” phase to see how your model is performing. Testing is an iterative process... ... you’ll run the test… …update the model… …and run the test again. This iterative process will help you develop your model at the start… …and refine it when it’s in use. The test set management feature will help your NLU apps perform at their best to give users the best experience. For more information, see our product documentation
or knowledge base. Or ask a question in the ServiceNow Community.