1、Use Conversational AI to Realize Insights As AService with Big DAbstract Gain an overview of building a Bot using various Microsoft products Learn how to get insights from complex data using Conversational AI Describe the challenges of domain-specific chat bots Overcome the challenges using latest M
2、SR technologies (Conversation Learner)Insight as a service- What s the use case scenarioProblem Inefficient Typical Insights Generation ProcessI need to knowcompanies with thehighest PropensityASAPInteresting, whatabout Cloud Usage ?Thats not what IaskedNike, Accenture,BMW, City Bank,GM, Dell * Fake
3、 answer How can we empower the field, marketing, strategy teamsin making data driven decisions by democratizing data andinsights in a timely fashion ?Situation Build an easy to use conversational engine with real-timeanalysis that can answer questions in the commercialsegment.Recommendation Build a
4、repository of internal and external data sourcesthat can be leveraged to generate insightsProof of concept Microsofts Mycroft AI that returns data + deep insightsthereby abstracting the analysis away from the user.Conversational AI: Azure Bot Service + Cognitive ServicesUser InputChannelsBot Intelli
5、genceBusiness ProcessAzure ToolsSecurityLoggingAuditingIntegrationA great bot provides a delightful user experienceBot Use CasesCommerce ChatbotCreate conversational interfaces for variousscenarios like banking, travel, and entertainment.Information ChatbotAnswer questions in a knowledge set or FAQ
6、usingQnA Maker and answer more open-ended questionsusing Azure Search.Enterprise Productivity ChatbotStreamline common work activities by integratingexternal systems.Solution ArchitectureStandard Data Access- Azure SQL- ElasticSearch- Impala- KustoWhat is the bot framework? Bot builder SDK + bot ser
7、viceC# Tools for building REST websites Services to enrich (LUIS, QnA, Azure Search, etc.) Data to debug and tools to analyzehttps:/ is the LUIS? Language Understanding (LUIS) is a cloud-based API service that applies custom machine-learning intelligence to a usersconversational, natural language te
8、xt to predict overall meaning, and pull out relevant, detailed information.https:/ page Identity & authorization Typical question templatesCompany A?Company A?Company B?Company C?Company D?All company names are fake/blanked in the following demo slides.Intro page Identity & authorization Typical que
9、stion templatesCompany A?Company A?Company B?Company C?Company D?All company names are fake/blanked in the following demo slides. More question template relatedto Cloud deployments, cloud andOS compete, feedback, M365,etc.Tell me about company MADWhat are the feedbacks from A? Extracted representati
10、vefeedbacks Word cloudWhat about deployments?HooliUmbrella CorporationInitechGlobex CorporationAcme CorporationFake Company AFake Company BWhat is the OS distribution about A?What are the top 4 companies with highest propensity score?InitechFake Company BMassive Dynamic(Fringe)HooliHow about some de
11、ep insights?However, things are not as Challenge 1: Fail To Ask Clarification QuestionsIn the dialog below, we can see that a new user is trying to ask question that the NLP engine was not able to catch.Ideally, the bot should have asked clarifying questions.DateAccessTime UserNameQueryReturnStatus
12、ErrorTypeDialog10/2/2017 3:46:53 PMHow many windows 10 devices are at AB InbevHow many windows 10 deployments are at AB inbev?What is the deployments for Pfizer?ERRORERRORSUCCESSNLP17517517510/2/2017 3:47:14 PM10/2/2017 3:48:16 PMEntity ExtractionChallenge 2: Untrained Questions This user started wi
13、th a good question that was not trained on The bot should have ask more clarifying questions based on the NLP classification, and retrained itself.DateAccessTime9:50:29 PM9:50:50 PMUserNameQueryReturnStatusERRORErrorTypeDialog10/10/201710/10/2017who has the most deploymentsNLP154154What is the top 1
14、0 company with the highest deployments?SUCCESSHow did we overcome the challenges?Conversation Learner : AI-driven task-oriented botsTraditional approachescity = nullNeuralnetworkWhich city?Solution: Conversation Learner1 + Entity Tracking+ Context SwitchingEntity extractor can be customizedBuilt-in
15、LUISextractorUser InputOur ExtractorLSTM ClassifierAction Entity detection plays a critical role in our We leverage the built-in entity extractorsystemfrom LUIS. But it is not sufficient. Extraction accuracy of entity values (pany names) directly impacts theperformance of the bot We dont want to tra
16、in the NLP with 50Kcompany names We build a company name extractor thatrecognizes the normalized company namefrom the utterance Domain-Specific Entity extraction is challenging Lots of instances in our Business Insightscenario Lots of variations on company namesacross systems Customers can upload a
17、dictionary forthe extractorTranslation VS ClassificationMany existing methods treat multi-rounddialog chatbot as a translation problem Natural language processing Parsing treeHelpBye Translate into SQL queryBI Bot treats Q&A as a user intentclassification problemMetric Map utterance to intentLSTM Fi
18、ll the predefined SQL template in each intentbased on detected entity valuesCorrelationAggregationContext Switch We have 15 actions that the LSTM is classifyingbased on user utterance and entities Including one for context switchingExtending Conversation Learner to our caseMulti-tasks inone Chat Bot
19、Context switchContext Switch DetectedWhat is the MAD for Accenture?What is the MAD for Accenture?What about in China?Show me Feedback in ChinaForget contextWhat is the MAD forChinaNo Context Switch DetectedWhat is the MAD for Accenture?Can you show me User Feedback aswell?What is the userfeedback in
20、 ChinaAnd what about IBM?Remember MADConversation Learner: Train It To Adapt To Mycroft ScenarioInteractiveteachingCLDeveloper plays roleDeveloper makescorrectionsof userMakecorrections tologged dialogsCLCLEnd usersDeveloper makescorrectionsAutonomousimprovementReinforcementlearningEnd usersWe Use U
21、ser-Study to Measure SuccessPseudo-experiment: Multi-round user study to assess improvementMetricOld Version New Version% of questions result in “I dontknow”46.2%9.1%90.9%N/A% of questions result in an answer 53.8% of answers that are marked ashelpful by the user6.2%Synthetic dialogsgeneratorTrained
22、 on Synthetic DialoguesLabeled Entity TypesLearnings Insights-As-A-Service can largely reduce time cost for ad-hoc reporting requests Domain knowledge is very valuable for building bot use it Whenever you want to start something from scratch, know that there is probablysome existing solutions that can accelerate your progress
