What’s up Watson? Using IBM Watson’s QAAPI with Bluemix, NodeExpress – Part 1


Published in IBM developerWorks ‘Whats up Watson? Using Watson QAAPI with Bluemix and NodeExpress

In this post I take the famed IBM Watson through the paces (yes, that’s right!, this post is about  using the same  IBM  Watson which trounced 2 human Jeopardy titans in a classic duel in 2011).  IBM’s Watson (see  What is Watson?) is capable of understanding the nuances of the English language and heralds a new era in the domain of cognitive computing. IBM Bluemix now includes 8 services from Watson ranging from Concept Expansion, Language Identification, Machine Translation, Question-Answer etc. For more information on Watson’s QAAPI and the many services that have been included in Bluemix please see Watson Services.

In this article I create an application on IBM Bluemix and use Watson’s QAAPI (Question-Answer API) as a service to the Bluemix application. For the application I have used NodeExpress to create a Webserver and post the REST queries to Watson.  Jade is used format the results of Watson’s Response.

In this current release of Bluemix Watson comes with a corpus of medical facts. In other words Watson has been made to ingest medical documents in multiple formats (doc, pdf, html, text  etc) and the user can pose medical questions to Dr.Watson. In its current avatar, its medical diet consisted of dishes from (CDC Health Topics, National Heart, Lung, and Blood Institute (NHLBI) National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS), National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), National Institute of Neurological Disorders and Stroke (NINDS), Cancer.gov (physician data query) etc.)

Try out my Watson app  on Bluemix here –  Whats up Watson?

To get down to Watson QAAPI business with Bluemix app you can fork the code from Devops at whatsup. This can then be downloaded to your local machine. You can also clone the code from GitHub at whatsup

  1. To get started go to the directory where you have cloned the code for Whatsup app

2.Push the app to Bluemix using Cloud Foundry’s ‘cf’ commands as shown below

cf login -a https://api.ng.bluemix.net

3. Next push the app to Bluemix
cf push whatsup –p . –m 512M

In the Bluemix dashboard you should see ‘whatsup’ app running. Now click ‘Add Service’ and under Watson add ‘Question Answer’

1

Add Qatson QAAPI

2

You will be prompted with ‘Restage Application’. Click ‘Ok’. Once you have the app running you should be able to get started with Doc Watson.

The code for this Bluemix app with QAAPI as a Service is based on the following article Examples using the Question and Answer API

  1. Here’s a look at the code for the Bluemix & Watson app.

In this Bluemix app I show the different types of Questions we can ask Watson and the responses we get from it. The app has a route for each of the different types of questions and options

a. Simple Synchronous Query: Post a simple synchronous query to Watson
This is the simplest query that we can pose to Watson. Here we need to just include the text of the question and the also a Sync Timeout. The Sync Timeout denotes the time client will wait for responses from the Watson service
// Ask Watson a simple synchronous query

app.get('/question',question.list);
app.post('/simplesync',simplesync.list);

b. Evidence based question: Ask Watson to respond to evidence given to it
Ask Watson for responses based on evidence given like medical conditions etc. This would be a used for diagnostic purposes I would presume.
// Ask Watson for responses based on evidence provided
app.get('/evidence',evidence.list);
app.post('/evidencereq',evidencereq.list);

c. Request for a specified set of answers to a question: Ask Dr. Watson to give a specified number of responses to a question
// Ask Watson to provide specified number of responses to a query
app.get('/items',items.list);
app.post('/itemsreq',itemsreq.list);

d. Get a formatted response to a question: Ask Dr. Watson to format the response to the question
// Get a formatted response from Watson for a query
app.get('/format',format.list);
app.post('/formatreq',formatreq.list);

  1. To get started with Watson we would need to connect the Bluemix app to the Watson’s QAAPI as a service by parsing the environment variable. This is shown below

//Get the VCAP environment variables to connect Watson service to the Bluemix application

question.js
o o o
if (process.env.VCAP_SERVICES) {
var VCAP_SERVICES = JSON.parse(process.env.VCAP_SERVICES);
// retrieve the credential information from VCAP_SERVICES for Watson QAAPI
var hostname   = VCAP_SERVICES["Watson QAAPI-0.1"][0].name;
var passwd = VCAP_SERVICES["Watson QAAPI-0.1"][0].credentials.password;
var userid = VCAP_SERVICES["Watson QAAPI-0.1"][0].credentials.userid;
var watson_url = VCAP_SERVICES["Watson QAAPI-0.1"][0].credentials.url;

Next we need to format the header for the POST request

var parts = url.parse(watson_url);
// Create the request options to POST our question to Watson
var options = {host: parts.hostname,
port: 443,
path: parts.pathname,
method: 'POST',
headers: headers,
rejectUnauthorized: false, // ignore certificates
requestCert: true,
agent: false};

The question that is to be asked of Watson needs to be formatted appropriately based on the input received in the appropriate form (for e.g. simplesync.jade)

question.js
// Get the values from the form
var syncTimeout = req.body.timeout;
var query = req.body.query;
// create the Question text to ask Watson
var question = {question : {questionText :query }};
var evidence = {"evidenceRequest":{"items":1,"profile":"yes"}};
// Set the POST body and send to Watson
req.write(JSON.stringify(question));
req.write("\n\n");
req.end();

Now you POST the Question to Dr. Watson and receive the stream of response using Node.js’ .on(‘data’,) & .on(‘end’) shown below

question.js
…..
      var req = https.request(options, function(result) {
// Retrieve and return the result back to the client
result.on(“data”, function(chunk) {
output += chunk;
});

result.on('end', function(chunk) {
// Capture Watson's response in output. Parse Watson's answer for the fields
var results = JSON.parse(output);
res.render(
'answer', {
"results":results
});
});
});

The results are parsed and formatted displayed using Jade. For the Jade templates I have used a combination of Jade and inline HTML tags (Jade can occasionally be very stubborn and make you sweat quite a bit. So I took the easier route of inline HTML tagging. In a later post I will try out CSS stylesheets to format the response.)

Included below is the part of the jade template with inline HTML tagging

Answer.jade
o o o
<h2 style="color:blueviolet">  Question Details </style> </h2>
for result in results.question.qclasslist
p <font color="blueviolet">  Value   = <font color="black "> #{result.value} </font>
p <font color="blueviolet">  Focuslist  </font> = <font color="black "> #{results.question.focuslist[0].value} </font>
// The 'How' query's response does not include latlist. Hence conditional added.
if latlist
p <font color="blueviolet">  Latlist  </font> = <font color="black "> #{results.question.latlist[0].value} </font>

o o o

Now that the code is all set you can fire the Watson. To do this click on the route

3

Click the route whatsup.mybluemix.net and ‘Lo and behold’ you should see Watson ready and raring to go.

4

As the display shows there are 4 different Question-Answer options that there is for Watson QAAPI

Simple Synchronous Question-Answer
This option is the simplest option. Here we need to just include the text of the question and the also a Sync Timeout. The question can be any medical related question as Watson in its current Bluemix avatar has a medical corpus

For e.g.1) What is carotid artery disease?

2) What is the difference between hepatitis A and hepatitis B etc.

The Sync Timeout parameter specifies the number of seconds the QAAPI client will wait for the streaming response from Watson. An example question and Watson’s response are included below
5
;

When we click Submit Watson spews out the following response

6

Evidence based response:

In this mode of operation, questions can be posed to Watson based on observed evidence. Watson will output all relevant information based on the evidence provided. As seen in the output Watson provides a “confidence factor” for each of its response

7

Watson gives response with appropriate confidence values based on the given evidence

8

Question with specified number of responses
In this option we can ask Watson to provide us with at least ‘n’ items in its response. If it cannot provide as many items it will give an error notification

11

This will bring up the following screen where the question asked is “What is the treatment for Down’s syndrome?” and Items as 3.
9

Watson gives 3 items in the response as shown below
10

Formatted Response: Here Watson gives a formatted response to question asked. Since I had already formatted the response using Jade it does not do extra formatting as seen in the screen shot.
12

Updated synonym based response. In this response we can change the synonym list based on which Watson will search its medical corpus and modify its response. The synonym list for the the question “What is fever?” is shown below. We can turn off synonyms by setting to ‘false’ and possibly adding other synonyms for the search
13

This part of the code has not been included in this post and is left as an exercise to the reader 🙂

As mentioned before you can fork and clone the code from IBM devops at whatsup or clone from GitHub at whatsup

There are many sections to Watson’s answer which cannot be included in this post as the amount of information is large and really needs to be pared to pick out important details. I am including small sections from each part of Watson’s response below to the question “How is carotid artery disease treated/”

I will follow up this post with another post where I will take a closer look at Watson’s response which has many parts to it
namely

– Question Details
14

– Evidence list
15

– Synonym list
16

– Answers

17

– Error notifications
18

There you have it.  Go ahead and get all your medical questions answered by Watson.

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

Other posts on Bluemix

1. Brewing a potion with Bluemix, PostgreSQL & Node.js in the cloud
2. A Bluemix recipe with MongoDB and Node.js
3. A Cloud Medley with IBM’s Bluemix, Cloudant and Node.js
4.  Bend it like Bluemix, MongoDB with autoscaling – Part 1

You may also find the following interesting
1. Informed choices through Machine Learning : Analyzing Kohli, Tendulkar and Dravid
2. Informed choices through Machine Learning-2: Pitting together Kumble, Kapil,
3. A crime map of India in R: Crimes against women
4. Analyzing cricket’s batting legends – Through the mirage with R


Find me on Google+

Advertisements

45 thoughts on “What’s up Watson? Using IBM Watson’s QAAPI with Bluemix, NodeExpress – Part 1

  1. Pingback: Revisiting Whats up, Watson – Using Watson’s Question and Answer with Bluemix – Part 2 | Giga thoughts …

  2. Pingback: Analyzing cricket’s batting legends – Through the mirage with R | Giga thoughts …

  3. Pingback: A crime map of India in R – Crimes against women | Giga thoughts …

  4. Pingback: A peek into literacy in India: Statistical Learning with R | Giga thoughts …

  5. Pingback: Informed choices through Machine Learning – Analyzing Kohli, Tendulkar and Dravid | Giga thoughts …

  6. Pingback: Working with Node.js and PostgreSQL | Giga thoughts …

  7. Pingback: Masters of Spin: Unraveling the web with R | Giga thoughts …

  8. Pingback: Informed choices through Machine Learning-2: Pitting together Kumble, Kapil, Chandra | Giga thoughts …

  9. Pingback: Re-working the Lucy Richardson algorithm in OpenCV | Giga thoughts …

  10. Pingback: Hand detection through Haartraining: A hands-on approach | Giga thoughts …

  11. Pingback: Experiments with deblurring using OpenCV | Giga thoughts …

  12. Pingback: De-blurring revisited with Wiener filter using OpenCV | Giga thoughts …

  13. Pingback: Deblurring with OpenCV: Weiner filter reloaded | Giga thoughts …

  14. Pingback: Powershell GUI – Adding bells and whistles | Giga thoughts …

  15. Pingback: Mirror, mirror … the best batsman of them all? | Giga thoughts …

  16. Pingback: Thinking Web Scale-1: Map-Reduce – Bring compute to data | Giga thoughts …

  17. Pingback: TWS-4: Gossip protocol: Epidemics and rumors to the rescue | Giga thoughts …

  18. Pingback: Into the Telecom vortex | Giga thoughts …

  19. Pingback: Installing and using OpenCV with Visual Studio 2010 express | Giga thoughts …

  20. Pingback: Adventures in LogParser, HTA and charts | Giga thoughts …

  21. Pingback: Tips for building a decent HTA/HTML application | Giga thoughts …

  22. Pingback: Stir fry a VBA with Excel application quickly | Giga thoughts …

  23. Pingback: Building a respectable VBA with Excel Application | Giga thoughts …

  24. Pingback: Get your feet wet with Powershell GUI | Giga thoughts …

  25. Pingback: Google’s Page Rank: Predicting the movements of a random web walker | Giga thoughts …

  26. Pingback: The common alphabet of programming languages | Giga thoughts …

  27. Pingback: The mind of a programmer | Giga thoughts …

  28. Pingback: How to program – Some essential tips | Giga thoughts …

  29. Pingback: Programming Zen and now – Some essential tips-2 | Giga thoughts …

  30. Pingback: Introducing cricketr! : A R package to analyze performances of cricketers | Giga thoughts …

  31. Pingback: Taking cricketr for a spin – Part 1 | Giga thoughts …

  32. Pingback: cricketr digs the Ashes! | Giga thoughts …

  33. Pingback: cricketr plays the ODIs! | Giga thoughts …

  34. Pingback: cricketr adapts to the Twenty20 International! | Giga thoughts …

  35. Pingback: Natural language processing: What would Shakespeare say? | Giga thoughts …

  36. Pingback: Revisiting crimes against women in India | Giga thoughts …

  37. Pingback: Literacy in India – A deepR dive | Giga thoughts …

  38. Pingback: Sixer – R package cricketr’s new avatar | Giga thoughts …

  39. Pingback: The making of cricket package yorkr – Part 3 | Giga thoughts …

  40. Pingback: Introducing cricket package yorkr: Part 1- Beaten by sheer pace! | Giga thoughts …

  41. Pingback: yorkr pads up for the Twenty20s: Part 1- Analyzing team”s match performance | Giga thoughts …

  42. Pingback: Re-introducing cricketr! : An R package to analyze performances of cricketers | Giga thoughts …

  43. Pingback: Venturing into IBM’s Quantum Computing Experience | Giga thoughts …

  44. Pingback: IBM Data Science Experience:  First steps with yorkr | Giga thoughts …

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s