Sixer – R package cricketr’s new Shiny avatar


Published in R-bloggers: Sixer – R package cricketr’s new Shiny app

In this post I create a Shiny App, Sixer, based on my R package cricketr. I had developed the R package cricketr, a few months back for analyzing the performances of batsman and bowlers in all formats of the game (Test, ODI and Twenty 20). This package uses the statistics info available in ESPN Cricinfo Statsguru. I had written a series of posts using the cricketr package where I chose a few batsmen, bowlers and compared their performances of these players. Here I have created a complete Shiny app with a lot more players and with almost all the features of the cricketr package. The motivation for creating the Shiny app was to

  • To show case the  ‘cricketr’ package and to highlight its functionalities
  • Perform analysis of more batsman and bowlers
  • Allow users to interact with the package and to allow them to try out the different features and functions of the package and to also check performances of some of their favorite crickets

a) You can try out the interactive  Shiny app Sixer at – Sixer
b) The code for this Shiny app project can be cloned/forked from GitHub – Sixer
Note: Please be mindful of  ESPN Cricinfo Terms of Use.
(Take a look at my short video tutorial on my R package cricketr on Youtube – R package cricketr – A short tutorial)

Check out my 2 books on cricket, a) Cricket analytics with cricketr b) Beaten by sheer pace – Cricket analytics with yorkr, now available in both paperback & kindle versions on Amazon!!! Pick up your copies today!

In this Shiny app I have 5 tabs which perform the following function
1.  Analyze Batsman
This tab analyzes batsmen based on different functions and plots the performances of the selected batsman. There are functions that compute and display batsman’s run-frequency ranges, Mean Strike rate, No of 4’s, dismissals, 3-D plot of Runs scored vs Balls Faced and Minutes at crease, Contribution to wins & losses, Home-Away record etc. The analyses can be done for Test cricketers, ODI and Twenty 20 batsman. I have included most of the Test batting giants including Tendulkar, Dravid, Sir Don Bradman, Viv Richards, Lara, Ponting etc. Similarly the ODI list includes Sehwag, Devilliers, Afridi, Maxwell etc. The Twenty20 list includes the Top 10 Twenty20 batsman based on their ICC rankings

2. Analyze bowler
This tab analyzes the bowling performances of bowlers, Wickets percentages, Mean Economy Rate, Wickets at different venues, Moving average of wickets etc. As earlier I have all the Top bowlers including Warne, Muralidharan, Kumble- the famed Indian spin quartet of Bedi, Chandrasekhar, Prasanna, Venkatraghavan, the deadly West Indies trio of Marshal, Roberts and Holding and the lethal combination of Imran Khan, Wasim Akram and Waqar Younis besides the dangerous Dennis Lillee and Jeff Thomson. Do give the functions a try and see for yourself the performances of these individual bowlers

3. Relative performances of batsman
This tab allows the selection of multiple batsmen (Test, ODI and Twenty 20) for comparisons. There are 2 main functions Relative Runs Frequency performance and Relative Mean Strike Rate

4. Relative performances of bowlers
Here we can compare bowling performances of multiple bowlers, which include functions Relative Bowling Performance and Relative Economy Rate. This can be done for Test, ODI and Twenty20 formats

5. Check for In-Form status of players
This tab checks the form status of batsman or bowler selected for all of the different formats of the game. The below computation uses Null Hypothesis testing and p-value to determine if the batsman is in-form or out-of-form. For this 90% of the career runs is chosen as the population and the mean computed. The last 10% is chosen to be the sample set and the sample Mean and the sample Standard Deviation are calculated. Note: The accuracy of the p-value test depends on the size of the population and the size of the sample set. It  may not be very significant for players with a few innings played.

Some of my earlier posts based my R package cricketr are listed below
1. Introducing cricketr!: An R package for analyzing performances of cricketers
2. Taking cricketr for a spin – Part 1
3. cricketr plays the ODIs
4. cricketr adapts to the Twenty20 International
5. cricketr digs the Ashes

Do try out the interactive Sixer Shiny app – Sixer
You can clone the code from Github – Sixer

There is not much in way of explanation. The Shiny app’s use is self-explanatory. You can choose a match type ( Test,ODI or Twenty20), choose a batsman/bowler  from the drop down list and select the plot you would like to seeHere a few sample plots
A. Analyze batsman tab
i) Batsman – Brian Lara , Match Type – Test, Function – Mean Strike Rate
sxr-1ii) Batsman – Shahid Afridi, Match Type –  ODI, Function – Runs vs Balls faced
The plot below shows that if Afridi faces around 50 balls he is likely to score around 60 runs in ODIs.
sxr-2iii)   Batsman – Chris Gayle, Match Type – Twenty20  Function – Moving Average
sxr-3B. Analyze bowler tab

i. Bowler – B S Chandrasekhar, Match Type – Test, Function – Wickets vs Runs
sxr-4ii)  Bowler – Malcolm Marshall, Match Type – Test, Function – Mean Economy Ratesxr-5iii)  Bowler – Sunil Narine, Match Type – Twenty 20, Function – Bowler Wicket Rate
sxr-6
C. Relative performance of batsman (you can select more than 1)
The below plot gives the Mean Strike Rate of batsman. Viv Richards, Brian Lara, Sanath Jayasuriya and David Warner are best strikers of the ball.
sxr-7

Here are some of the great strikers of the ball in ODIs
sxr-8D. Relative performance of bowlers (you can select more than 1)
Finally a look at the famed Indian spin quartet.  From the plot below it can be seen that  B S Bedi  & Venkatraghavan were more economical than Chandrasekhar and Prasanna.
sxr-9

But the latter have a better 4-5 wicket haul than the former two as seen in the plot below

sxr-11Finally a look at the average number of balls to take a wicket by the Top 4 Twenty 20 bowlers.
sxr-10

E. Check for In-form status of players
Note: The accuracy of the p-value depends on the size of the population and the size of the sample set. It  may not be very significant for smaller data sizes

i. Match Type – Test,  Player Type – Batsman  Name – Wickets v
In this plot the in-form status of Viv Richards is checked. This shows that Viv Richards was out-of-form
sxr-12In the plot below the form status of S. Venkataraghavan is shown. According to this at the time of  his retirement S Venkat was still in-form
sxr-13

Do give the Shiny app Sixer a try.

Also see
1. Literacy in India : A deepR dive.
2.  Natural Language Processing: What would Shakespeare say?
3. Revisiting crimes against women in India
4. Informed choices through Machine Learning : Analyzing Kohli, Tendulkar and Dravid
5. Experiments with deblurring using OpenCV
6.  What’s up Watson? Using IBM Watson’s QAAPI with Bluemix, NodeExpress – Part 1
7.  Working with Node.js and PostgreSQL
8. A method for optimal bandwidth usage by auctioning available bandwidth using the OpenFlow Protocol
9.  Latency, throughput implications for the cloud
10.  A closer look at “Robot horse on a Trot! in Android”

Advertisements

Bend it like Bluemix, MongoDB with auto-scaling – Part 3


In this last post of this series, I test the performance of Bluemix & MongoDB against  concurrent queries and deletes to the cloud based app with Mongo DB, with auto-scaling on. Before I started these series of tests I moved the Overload policy a couple of notches higher and made it scale out if  memory utilization > 75% for 120 secs and < 30% for 120 secs (from the earlier 55% memory utilization) as shown below.

27

The code for bluemixMongo app can be forked from Devops at bluemixMongo or can be cloned from GitHub at  bluemix-mongo-autoscale. The multi-mechanize scripts can be downloaded from GitHub at multi-mechanize     Before starting the testing I checked the current number of documents inserted by the concurrent inserts (see Bend it like Bluemix., MongoDB using Auto-scaling – Part 2). The total number as determined by checking the logs was 1380 17   Sure enough with the scaling policy change after 2 minutes the number of instanced dropped from 3 to 2

18

1. Querying the bluemixMongo app with Multi-mechanize

The Multi-mechanize Python script used for querying the bluemixMongo app simply invokes the app’s userlist URL (resp=br.open(“http://bluemixmongo.mybluemix.net/userlist/&#8221;)

v_user.py

def run(self):
# create a Browser instance
br = mechanize.Browser()
# don"t bother with robots.txt
br.set_handle_robots(False)
# start the timer
start_timer = time.time()
#print("Display userlist")
# Display 5 random documents
resp=br.open("http://bluemixmongo.mybluemix.net/userlist/")
assert("Example Mongo Page" in resp.get_data())
# stop the timer
latency = time.time() - start_timer
self.custom_timers["Userlist"] = latency
r = random.uniform(1, 2)
time.sleep(r)
self.custom_timers['Example_Timer'] = r

The configuration setup for this script creates 2 sets of 10 concurrent threads

config.cfg
run_time = 300
rampup = 0
results_ts_interval = 10
progress_bar = on
console_logging = off
xml_report = off
[user_group-1]
threads = 10
script = v_user.py
[user_group-2]
threads = 10
script = v_user.py

The corresponding userlist.js for querying the app is shown below. Here the query is constructed by creating a ‘RegularExpression’ with  a random Firstname, consisting of a random letter and a random number. Also the query is also limited to 5 documents.

function(callback)
{
// Display a random set of 5 records based on a regular expression made with random letter, number
var randnum = Math.floor((Math.random() * 10) + 1);
var alpha = ['A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','X','Y','Z'];
var randletter = alpha[Math.floor(Math.random() * alpha.length)];
var val =  randletter + ".*" + randnum + ".*";
// Limit the display to 5 documents
var results = collection.find({"FirstName": new RegExp(val)}).limit(5).toArray(function(err, items){
if(err) {
console.log(err + " Error getting items for display");
}
else {
res.render('userlist',
{ "userlist" : items
}); // end res.render
} //end else
db.close(); // Ensure that the open connection is closed
}); // end toArray function
callback(null, 'two');
}

2. Running the userlist query

The following screenshot shows the userlist query being executed concurrently with Multi-mechanize. Note that the number of  instances also drops down to 1

18

3. Deleting documents with Multi-mechanize

The multi-mechanize script for deleting a document is shown below. This script calls the URL with resp = br.open(“http://bluemixmongo.mybluemix.net/remuser&#8221;). No values are required to be entered into the form and the ‘submit’ is simulated.

v_user.py
def run(self):
# create a Browser instance
br = mechanize.Browser()
# don"t bother with robots.txt
br.set_handle_robots(False)
br.addheaders = [("User-agent", "Mozilla/5.0Compatible")]
# start the timer
start_timer = time.time()
# submit the request
resp = br.open("http://bluemixmongo.mybluemix.net/remuser")
#resp = br.open("http://localhost:3000/remuser")
resp.read()
# stop the timer
latency = time.time() - start_timer
# store the custom timer
self.custom_timers["Load_Front_Page"] = latency
# think-time
time.sleep(2)
# select first (zero-based) form on page
br.select_form(nr=0)
# set form field
br.form["firstname"] = ""
br.form["lastname"] = ""
br.form["mobile"] = ""
# start the timer
start_timer = time.time()
# submit the form
resp = br.submit()
resp.read()
print("Removed")
# stop the timer
latency = time.time() - start_timer
# store the custom timer
self.custom_timers["Delete"] = latency
# think-time
time.sleep(2)

config.cfg

The config file is set to start 2 sets of 10 concurrent threads and execute for 300 secs

[global]
run_time = 300
rampup = 0
results_ts_interval = 10
progress_bar = on
console_logging = off
xml_report = off
[user_group-1]
threads = 10
script = v_user.py
[user_group-2]
threads = 10
script = v_user.py
;

deleteuser.js

This Node.js script does a findOne() document and does a remove with the ‘justOne’ set to true

collection.findOne(function(err, item) {
// Delete just a single record
collection.remove(item, {justOne:true},(function (err, doc) {
if (err) {
// If it failed, return error
res.send("There was a problem removing the information to the database.");
}
else {
// If it worked redirect to userlist
res.location("userlist");
// And forward to success page
res.redirect("userlist");
}
}));
});
collection.find().toArray(function(err, items) {
console.log("Length =----------------" + items.length);
db.close();
});
callback(null, 'two');

4. Running the deleteuser multimechanize script

The output of the script executing and the reduction of the number of instances because of the change in the memory utilization policy is shown

21

5. Multi-mechanize

As mentioned in the previous posts

The multi-mechanize commands are executed as follows
To create a new project
multimech-newproject.exe userlist
This will create 2 folders a) results b) test_scripts and the file c) config.cfg. The v_user.py needs to be updated as required

To run the script
multimech-run.exe userlist

The details of the response times for the query is shown below .

All_Transactions_response_times_intervals

 

More details on latency and throughput for the queries and the deletes are included in the results folder of multi-mechanize    

6. Autoscaling The details of the auto-scaling service is shown below

a. Scaling Metric Statistics

22

b. Scaling history 23

7. Monitoring and Analytics (M & A) The output from M & A is shown  below

 

a. Performance Monitoring 24  

b. Log Analysis output The log analysis give a detailed report on the calls made to the app, the console log output and other useful information

25

The series of the 3 posts Bend it like Bluemix, MongoDB with auto-scaling demonstrated the ability of the cloud to expand and shrink based on the load on the cloud.An important requirement for Cloud Architects is design applications that can  scale horizontally without impacting the performance while keeping the costs optimum. The real challenge to auto-scaling is the need to make the application really distributed as opposed to the monolithic architectures we are generally used to.   I hope to write another post on creating simple distributed application later.

Hasta la Vista!

Also see
1.  Bend it like Bluemix, MongoDB with autoscaling – Part 1
2. Bend it like Bluemix, MongoDB with autoscaling – Part 2

You may like :
a) Latency, throughput implications for the cloud
b) The many faces of latency
c) Brewing a potion with Bluemix, PostgreSQL & Node.js in the cloud
d)  A Bluemix recipe with MongoDB and Node.js
e)Spicing up IBM Bluemix with MongoDB and NodeExpress
f) Rock N’ Roll with Bluemix, Cloudant & NodeExpress

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

Bend it like Bluemix, MongoDB using Auto-scaling – Part 2!


This post takes off from my previous post Bend it like Bluemix, MongoDB using Auto-scale –  Part 1! In this post I generate traffic using Multi-Mechanize a performance test framework and check out the auto-scaling on Bluemix, besides also doing some rudimentary check on the latency and throughput for this test application. In this particular post I generate concurrent threads which insert documents into MongoDB.

Note: As mentioned in my earlier post this is more of a prototype and the typical situation when architecting cloud applications. Clearly I have not optimized my cloud app (bluemixMongo) for maximum efficiency. Also this a simple 2 tier application with a rudimentary Web interface and a NoSQL DB at This is more of a Proof of Concept (PoC) for the auto-scaling service on Bluemix.

As earlier mentioned the bluemixMongo app is a modification of my earlier post Spicing up a IBM Bluemix cloud app with MongoDB and NodeExpress. The bluemixMongo cloud app that was used for this auto-scaling test can be forked from Devops at bluemixMongo or from GitHib at bluemix-mongo-autoscale. The Multi-mechanize config file, scripts and results can be found at GitHub in multi-mechanize

The document to be inserted into MongoDB consists of 3 fields – Firstname, Lastname and Mobile. To simulate the insertion of records into MongoDB I created a Multi-Mechanize script that will generate random combination of letters and numbers for the First and Last names and a random 9 digit number for the mobile. The code for this script is shown below

1. The snippet below measure the latency for loading the ‘New User’ page

v_user.py
def run(self):
# create a Browser instance
br = mechanize.Browser()
# don"t bother with robots.txt
br.set_handle_robots(False)
print("Rendering new user")
br.addheaders = [("User-agent", "Mozilla/5.0Compatible")]
# start the timer
start_timer = time.time()
# submit the request
resp = br.open("http://bluemixmongo.mybluemix.net/newuser")
#resp = br.open("http://localhost:3000/newuser")
resp.read()
# stop the timer
latency = time.time() - start_timer
# store the custom timer
self.custom_timers["Load Add User Page"] = latency
# think-time
time.sleep(2)

The script also measures the time taken to submit the form containing the Firstname, Lastname and Mobile

# select first (zero-based) form on page
br.select_form(nr=0)
# Create random Firstname
a = (''.join(random.choice(string.ascii_uppercase) for i in range(5)))
b = (''.join(random.choice(string.digits) for i in range(5)))
firstname = a + b
# Create random Lastname
a = (''.join(random.choice(string.ascii_uppercase) for i in range(5)))
b = (''.join(random.choice(string.digits) for i in range(5)))
lastname = a + b
# Create a random mobile number
mobile = (''.join(random.choice(string.digits) for i in range(9)))
# set form field
br.form["firstname"] = firstname
br.form["lastname"] = lastname
br.form["mobile"] = mobile
# start the timer
start_timer = time.time()
# submit the form
resp = br.submit()
print("Submitted.")
resp.read()
# stop the timer
latency = time.time() - start_timer
# store the custom timer
self.custom_timers["Add User"] = latency

2. The config.cfg file is setup to generate 2 asynchronous thread pools of 10 threads for about 400 seconds

config.cfg
run_time = 400
rampup = 0
results_ts_interval = 10
progress_bar = on
console_logging = off
xml_report = off
[user_group-1]
threads = 10
script = v_user.py
[user_group-2]
threads = 10
script = v_user.py

3. The code to add a new user in the app (adduser.js) uses the ‘async’ Node module to enforce sequential processing.

adduser.js
async.series([
function(callback)
{
collection = db.collection('phonebook', function(error, response) {
if( error ) {
return; // Return immediately
}
else {
console.log("Connected to phonebook");
}
});
callback(null, 'one');
},
function(callback)
// Insert the record into the DB
collection.insert({
"FirstName" : FirstName,
"LastName" : LastName,
"Mobile" : Mobile
}, function (err, doc) {
if (err) {
// If it failed, return error
res.send("There was a problem adding the information to the database.");
}
else {
// If it worked, redirect to userlist - Display users
res.location("userlist");
// And forward to success page
res.redirect("userlist")
}
});
collection.find().toArray(function(err, items) {
console.log("**************************>>>>>>>Length =" + items.length);
db.close(); // Make sure that the open DB connection is close
});
callback(null, 'two');
}
]);

4. To checkout auto-scaling the instance memory was kept at 128 MB. Also the scale-up policy was memory based and based on the memory of the instance exceeding 55% of 128 MB for 120 secs. The scale up based on CPU utilization was to happen when the utilization exceed 80% for 300 secs.

6

5. Check the auto-scaling policy

7

6. Initially as seen there is just a single instance

9

7. At around 48% of the script with around 623 transactions the instance is increased by 1. Note that the available memory is decreased by 640 MB – 128 MB = 512 MB.

10

8. At around 1324 transactions another instance is added

Note: Bear in mind

a) The memory threshold was artificially brought down to 55% of 128 MB.b) The app itself is not optimized for maximum efficiency

12

9. The Metric Statistics tab for the Autoscaling service shows this memory breach and the trigger for autoscaling

13

10. The Scaling history Tab for the Auto-scaling service displays the scale-up and scale-down and the policy rules based on which the scaling happened

14

11. If you go to the results folder for the Multi-mechanize tool the response and throughput are captured.

The multi-mechanize commands are executed as follows
To create a new project
multimech-newproject.exe adduser
This will create 2 folders a) results b) test_scripts and the file c) config.cfg. The v_user.py needs to be updated as required

To run the script
multimech-run.exe adduser

12.The results are shown below

a) Load Add User page (Latency)

Load Add User Page_response_times_intervals

b) Load Add User (Throughput)

Load Add User Page_throughput

c)Load Add User (Latency)

Add User_response_times_intervals

d) Load Add User (Throughput)

Add User_throughput

The detailed results can be seen at GitHub at multi-mechanize

13. Check the Monitoring and Analytics Page

a) Availability

16

b) Performance monitoring

15

So once the auto-scaling happens the application can be fine-tuned and for performance. Obviously one could do it the other way around too.

As can be seen adding NoSQL Databases like MongoDB, Redis, Cloudant DB etc. Setting up the auto-scaling policy is also painless as seen above.

Of course the real challenge in cloud applications is to make them distributed and scalable while keeping the applications themselves lean and mean!

See also

Also see
1.  Bend it like Bluemix, MongoDB with autoscaling – Part 1
3. Bend it like Bluemix, MongoDB with autoscaling – Part 3

You may like :
a) Latency, throughput implications for the cloud
b) The many faces of latency
c) Brewing a potion with Bluemix, PostgreSQL & Node.js in the cloud
d)  A Bluemix recipe with MongoDB and Node.js
e)Spicing up IBM Bluemix with MongoDB and NodeExpress
f) Rock N’ Roll with Bluemix, Cloudant & NodeExpress

a) Latency, throughput implications for the cloud

b) The many faces of latency

c) Design principles of scalable, distributed systems

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

Bend it like Bluemix, MongoDB using Auto-scale – Part 1!


In the next series of posts I turn on the heat on my cloud deployment in IBM Bluemix and check out the elastic nature of this PaaS offering. Handling traffic load and elastically expanding and contracting is what the cloud does best. This is  where the ‘rubber really meets the road”. In this series of posts I generate the traffic load using Multi –Mechanize a performance test framework created by Corey Goldberg.

This post is based on an earlier cloud app that I created on Bluemix namely Spicing up a IBM Bluemix Cloud app with MongoDB and NodeExpress. I had to make changes to this code to iron out issues while handling concurrent  inserts, displays and deletes issued from the multi-mechanize tool and also to manage the asynchronous nightmare of Nodejs.

The code for this Bluemix, MongoDB with Auto-scaling can be forked  from Devops at bluemixMongo. The code can also be cloned from GitHub at bluemix-mongo-autoscale

1.  To get started, fork the code from Devops at bluemixMongo. Then change the host name in manifest.yml to something unique and click the Build and Deploy button on the top right in the page.

26

1a.  Alternatively the code can be cloned from GitHub at bluemix-mongo-autoscale. From the directory where the code is cloned push the code using Cloud Foundry’s cf command as follows

cf login -a https://api.ng.bluemix.net

cf push bluemixMongo –p . –m 128M

2. Now add the MongoDB service and click ‘OK’ to restage the server.

3

3. Add the Monitoring and Analytics (M & A) and also the Auto-scaling service. The M& A gives a good report on the Availability, Performance logging, and also provides Logging Analysis. The Auto-scaling service is the service that allows the app to expand elastically to changing traffic loads.

4

4. You should see the bluemixMongo app running with 3 services MongoDB, Autoscaling and M&A

5

5. You should now be able click the bluemixMongo.mybluemix.net and check the application out.

6.Now you configure the Overload Policy (auto scaling) policy. This is a slightly contrived example and the scaling policy is set to scale up if the Memory exceeds 55%. (Typically the scale up would be configured for > 80% memory usage)

6

7. Now check the configured Auto-scaling policy

7

8. Change the Memory Quota as appropriate. In my case I have kept the memory quota as 128 MB. Note the available memory is 640 MB and hence allows up to 5 instances. (By the way it is also possible to set any other value like 100 MB).

5

9. Click the Monitoring and Analytics service and take a look at the output in the different tabs

8

10. Next you need to set up the Performance test tool – Multi mechanize. Multi-mechanize creates concurrent threads to generate the load on a Web site or service. It is based on Python which  makes it easy to modify the scripts for hitting a website, making a REST call or submitting a form.

To setup Multi-mechanize you also need additional packages like numpy  matplotlib etc as the tool generates traffic based on a user provided script, measures latency and throughput besides also generating graphs for these.

For a detailed steps for setup of Multi mechanize please follow the steps in Trying out multi-mechanize web performance and load testing framework. Note: I would suggest that you install Python 2.7.2 and not the later 3.x version as some of the packages are based on the 2.7 version which has a slightly different syntax for certain Python statements

In the next post I will run a traffic test on the bluemixMongo application using Multi-mechanize and observe how the cloud app responds to the load.

Watch this space!
Also see
Bend it like Bluemix, MongoDB with autoscaling – Part 2!
Bend it like Bluemix, MongoDB with autoscaling – Part 3

You may like :
a) Latency, throughput implications for the cloud
b) The many faces of latency
c) Brewing a potion with Bluemix, PostgreSQL & Node.js in the cloud
d)  A Bluemix recipe with MongoDB and Node.js
e)Spicing up IBM Bluemix with MongoDB and NodeExpress
f) Rock N’ Roll with Bluemix, Cloudant & NodeExpress

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

Revisiting Bluemix with Twilio


This post walks you through the steps to use Twilio with IBM’s Bluemix to send an SMS and also make a  voice call when you click a URL.  Twilio, is a cloud communications SaaS organization which allows you to use standard web languages to build voice, SMS and VOIP applications via a Web API.

Twilio provides the ability to build VOIP applications using APIs. Twilio itself resides in the cloud and is always available. It also provides SIP integration which means that it can be integrated with Soft switches. Twilio looks really interesting with its ability to combine the cloud, Web and VOIP, SMS and the like.

The steps given below allow you to use your app to perform 2 things by clicking the app’s URL namely websmstest.bluemix.net

a) Send a SMS to your mobile phone

b) Make a voice call to your mobile phone

The code can be forked from Devops at websmstest

Connecting Twilio with Bluemix

  1. Fire-up a Node.js Webstarter application from the Bluemix dashboard. In my case I have named the application websmstest. Once this is up and running

fig1

2) Click Add a Service and under ‘Web and Application’ and choose Twilio.

3) Enter a name for the Twilio service. You will also need the Account SID and Authorization token

  1. For this go to http://www.twilio.com and sign up

5) Once you have registered, go to your Twilio Dashboard for the Account SID and Auth Token. If the Auth token is encrypted, you can click the ‘lock’ symbol to display the Auth token in plain text.

  1. Enter the Account SID and Auth Token in the Twilio service in Bluemix in the right hand panel shown in the picture below

fig2

  1. To get started click the link websmstest code from Devops.

  2. Next click the ‘Edit Code’ button at the top

  3. Then click ‘Fork’ and provide a suitable name for your project

fig6

  1. Check the option for a) Deploy to Bluemix. Uncheck the other options a) Make it private b) Add features for Scrum development

  2. On the left hand side navigate to the file you need to edit and make the changes with the Devops GUI editor. You will need to make the following changes

Setup the application

12) You will need to modify the following files

  1. manifest.yml
  2. app.js

13) In the manifest.yml make sure you enter the name of your application and the host

applications:

- host: websmstest
  disk: 1024M
  name: websmstest
  command: node app.js
  path: .
  domain: mybluemix.net
  mem: 128M
  instances: 1

14) Lastly make changes to your app.js.

var app = require('gopher'),
    twilio = require('twilio');

 
var config = JSON.parse(process.env.VCAP_SERVICES);
 
var twilioSid, twilioToken;
config['user-provided'].forEach(function(service) {
    if (service.name == 'Twilio') {
        twilioSid = service.credentials.accountSID;
        twilioToken = service.credentials.authToken;
    }
});
 

// URL 
app.get('/', function(request, response) {
    var client = new twilio.RestClient(twilioSid, twilioToken);
 
    /* To make a voice call to your mobile phone uncomment the next 2 lines */
   //client.calls.create({
   //url: "http://twimlets.com/message?Message%5B0%5D=Hello",
   
    
     //  to: Enter your mobile phone  for e.g.98765 43210
     // from: Enter the number Twilio alloted to your account
     // body: The message you would like to send
     client.sendMessage({
    	  to: '+919876543210',
         from: '+16305476427',
         body:'Twilio notification through Bluemix!'
        }, function(err, message) {
        response.send('Message sent! ID:'+message.sid);
    });
});
  1. Enter your mobile number in the ‘to:’ line.

  2. Enter the number provided to you in your Twilio account see below

fig3

  1. In the app.js code above in step 14) use the green highlighted line to send a SMS to your mobile phone

  2. If you uncomment the blue highlighted lines a voice call will be made to your mobile

  3. Finally ‘Deploy’ the application on to Bluemix (more details on Deploying to Bluemix) can be found at Getting started with IBM Bluemix and IBM Devops services using Node.js

Test the application

19) Now click on your application to open the details and then click the link adjacent to the Routes.

fig8

20) You should see that an SMS has been sent as shown

fig4

21) Your mobile should now display the message that was sent as shown below

Screenshot_2014-06-22-13-41-44

22) Uncomment the lines which deal with making voice call and you should receive a voice announcement (see below) (Remember to comment the green highlighted line client.sendMessage!)

1

23) Check the analytics in your Twilio dashboard

fig5

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions


Find me on Google+

Rock N’ Roll with Bluemix, Cloudant & NodeExpress


Published as a tutorial in IBM Cloudant – Bluemix tutorial and demos

In this post I create a  cloud application that is based on Bluemix, Cloudant DB and NodeExpress.  NodeExpress is used to perform DB operations on  CloudantDB

The code can be forked from Devops from bluemix-cloudant-exp. You can also clone the code from Github from bluemix-cloudant-exp

The following NodeExpress routes are created for performing the basic database operations

  1. a) Displaying the list of books
  2. b) Adding a book
  3. c) Updating a book and
  4. d) Deleting a book

1

 

Push the app to Bluemix

a) Push the app to Bluemix using
cf push bluemix-cloudant -p . -m 512M

b) In the Bluemix dashboard add the Cloudant service.

c) Double click the CloudantNoSQLDB

9

and then click the ‘Launch’ button. This will bring the WebSQL based version of Cloudant DB

10

c) Next click the link bluemix-cloudant.mybluemix.net

11

This will start the Webserver and also populate the database.

e) This can be seen in the Cloudant Dashboard for the ‘test’ database which has 3 records shown below
12
The setup for these routes in the NodeExpress are as follows

app.get('/', routes.index);
app.get('/booklist', booklist.list);
app.get('/newbook', newbook.list);
app.post('/addbook',addbook.list);
app.get('/changebook', changebook.list);
app.post('/updatebook', updatebook.list);
app.get('/rembook', rembook.list);
app.post('/deletebook',deletebook.list);

Setting up the environment for Cloudant’s PouchDB

The first thing is to setup the environment for Cloudant’s Pouch DB by parsing the process.env environment variables as shown below

//Parse the process.env for the port and host that we've been assigned
if (process.env.VCAP_SERVICES) {
// Running on Bluemix. Parse the port and host that we've been assigned.
var env = JSON.parse(process.env.VCAP_SERVICES);
var host = process.env.VCAP_APP_HOST;
var port = process.env.VCAP_APP_PORT;
console.log('VCAP_SERVICES: %s', process.env.VCAP_SERVICES);
// Also parse Cloudant settings.
var cloudant = env['cloudantNoSQLDB'][0]['credentials'];
}
var db = new pouchdb('books'),
remote =cloudant.url + '/books';
opts = {
continuous: true
};
// Replicate the DB to remote
console.log(remote);
db.replicate.to(remote, opts);
db.replicate.from(remote, opts);

Displaying the list of books

Cloudant responds to DB queries as JSON messages. Hence to display the list of books the fields of each document is stored as an array and then displayed using the Jade table in booklist.jade  This is shown below

  1. a) booklist.js

var docs = db.allDocs(function(err, response) {
val = response.total_rows;
var details = "";
j=0;
var booklist = new Array(val);
for(i=0; i < val; i++) {
db.get(response.rows[i].id, function (err,doc){
j++;
booklist[j] = new Array(3);
booklist[j][0] = doc._id;
booklist[j][1] = doc.Title;
booklist[j][2] = doc.author;
details= details + JSON.stringify(doc.Title) + "  " +  JSON.stringify(doc.author) + "\n";
// Kludge because of Node.js asynchronous handling. To be fixed - T V Ganesh
if(j == val) {
res.render('booklist', {
"booklist" : booklist
});
}
}); // End db.get
} //End for
}); // End db.allDocs

  1. b) booklist.jade

The jade template simply displays the each booklist as a row in a table

block content
h1= "Display the list of books"
p
strong DocId  Title   Author
table
each book, i in booklist
tr
td #{book}
p
p
a(href='/') Home

2

Adding a book
To add a book the book details are obtained using the newbook.jade which display a form
block content
h1= "Add a book"
form#formAddBook(name="addbook",method="post",action="/addbook")
input#inputBookTitle(type="text", placeholder="Title", name="title")
input#inputBookAuthor(type="text", placeholder="Author", name="author")
button#btnSubmit(type="submit") submit
a(href='/') Home

With the values obtained from the form above a document is inserted into the books database as follows

// Get our form values. These rely on the "name" attributes
var Title = req.body.title;
var Author = req.body.author;
db.put({
author: Author,
Title : Title,
}, Title, function (err, response) {
console.log(err || response);
if (err) {
// If it failed, return error
res.send("There was a problem adding the information to the database.");
}
else {
// Redirect to booklist - Display booklist
res.location("booklist");
// And forward to success page
res.redirect("booklist");
}
});

Note: When inserting a document into the books database the docid for the document is set to be the same as the book Title itself

7

4

Updating a book

To update a document we need to input the document id. Also the document to be updated should use the “_rev” field which is obtained when we get the document. The values to be input are taken with the changeuser form

block content

h1= "Update a book"
form#formUpdateBook(name="addbook",method="post",action="/updatebook")
input#inputDocId(type="text", placeholder="DocId", name="docid")
input#inputBookTitle(type="text", placeholder="Title", name="title")
input#inputBookAuthor(type="text", placeholder="Author", name="author")
button#btnSubmit(type="submit") submit
a(href='/') Home

The values obtained are used to populate the document as follows

db.get(DocId, function(err, response) {
db.put({
_id: DocId,
_rev: response._rev,
author: Author,
Title : Title,
}, function(err, response) {
if (err) {
// If it failed, return error
res.send("There was a problem updating the information to the database.");
}
else {
// If it worked, redirect to display books
res.location("booklist");
// And forward to success page
res.redirect("booklist");
}
});
});

6

5

Deleting a document

To delete a document we need the document id which is taken with the rembook.jade form

block content
h1= "Delete a book"
form#formDeleteBook(name="addbook",method="post",action="/deletebook")
input#DocId(type="text", placeholder="DocId", name="docid")
button#btnSubmit(type="submit") submit
a(href='/') Home

//Deleting document book1
db.get(DocId, function(err, doc) {
db.remove(doc, function(err, response) {
if (err) {
// If it failed, return error
res.send("There was a problem removing the information to the database.");
}
else {
// Redirect to booklist
res.location("booklist");
// And forward to success page
res.redirect("booklist");
}
console.log(err || response);
});
});

In the diagram below docid ‘book3’ is deleted

13

8

Important tips

  1. If you run into issues while create a Jade template then do the following

npm install jade --g

You can check your jade template for correctness using

jade <name of jade template>

If the response is ‘rendered <name of jade template>.html’ then the template is fine.

  1. If there are problems with deploying the application or if the application crashes you check the cf logs as follows for the issue

cf logs <name of application> --recent

As mentioned the  code can be forked from Devops from bluemix-cloudant-exp. You can also clone the code from Github from bluemix-cloudant-exp

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

See also

1. Brewing a potion with Bluemix, PostgreSQL & Node.js in the cloud
2. A Bluemix recipe with MongoDB and Node.js
3. Spicing up IBM Bluemix with MongoDB and NodeExpress
4. A Cloud Medley with IBM’s Bluemix, Cloudant and Node.js
Find me on Google+

Spicing up a IBM Bluemix cloud app with MongoDB and NodeExpress


In this post I highlight the rudiments for a creating a cloud application on IBM’s PaaS offering Bluemix, using MongoDB and NodeExpress.   Clearly Bluemix allows one to fire up a cloud application with a NoSQL database in a matter of  a few hours which makes it really attractive. The NodeExpress  application was initially created using Enide Studio for Node.js  with a local Mongodb server running on my desktop. (Please see my post Elements of CRUD with Node Express and MongoDB) Once you have ironed out the issues in this local application you are ready to deploy on IBM Bluemix.

The code for this Bluemix application can be forked from bluemix-mongo from IBM Devops.

You can also clone the code from GitHub at bluemix-mongo

Here are the key changes that need to be made for running the NodeExpress Webserver with MongoDB as the backend DB

1) Webserver : Setup the port and host for the Webserver.

  1. app.js

var port = (process.env.VCAP_APP_PORT || 1337);
var host = (process.env.VCAP_APP_HOST || '0.0.0.0');
var app = express();
app.configure(function(){
app.set('port', port);

As seen above the host & port for the Webserver are obtained from the process.env variable.
2) Routes and Middleware
Setup the routes and invoke them appropriately in app.js
var express = require('express')
, routes = require('./routes')
, user = require('./routes/user')
, userlist = require('./routes/userlist')
, newuser = require('./routes/newuser')
, adduser = require('./routes/adduser')
, changeuser = require('./routes/changeuser')
, updateuser = require('./routes/updateuser')
, remuser = require('./routes/remuser')
, deleteuser = require('./routes/deleteuser')

app.get('/users', user.list);
app.get('/helloworld', routes.index);
app.get('/userlist', userlist.list);
app.get('/newuser', newuser.list);
app.post('/adduser',adduser.list);
app.get('/changeuser', changeuser.list);
app.post('/updateuser', updateuser.list);
app.get('/remuser', remuser.list);
app.post('/deleteuser',deleteuser.list);

3) Initialize MongoDB database: Create a set of 3 records when the Webserver starts as follows

if (process.env.VCAP_SERVICES) {
var env = JSON.parse(process.env.VCAP_SERVICES);
if (env['mongodb-2.2']) {
var mongo = env['mongodb-2.2'][0]['credentials'];
}
} else {
var mongo = {
"username" : "user1",
"password" : "secret",
"url" : "mongodb://user1:secret@localhost:27017/test"
}
}
var MongoClient = mongodb.MongoClient;
var db= MongoClient.connect(mongo.url, function(err, db) {
if(err) {
log("failed to connect to the database");
} else {
log("connected to database");
}
var collection = db.collection('phonebook');
//Clear DB and insert 3 records
remove(mycallback);
var user1 = { "FirstName" : "Tinniam", "LastName" : "Ganesh","Mobile": "916732177728" };
var user2 = { "FirstName" : "Darth", "LastName" : "Vader","Mobile": "6666699999" };
var user3 = { "FirstName" : "Bill", "LastName" : "Shakespeare","Mobile": "8342189991" };

  1. insert(user1,function(err,result){});
  2. insert(user2,function(err,result){});
  3. insert(user3,function(err,result){});
  4. find().toArray(function(err, items) {

});
});

3) Home Page: Setup up a Home page with the CRUD operations when the Bluemix cloud application’s route  for e.g. http://bluemix-mongo.mybluemix.net is clicked. This is shown below.

1

 

2

4) Display Users: To display the list of users the route /userlist is invoked. This function gets all the records from the collection and stores them into a toArray element, which is then used for rendering the list of uses with a ‘userlist.jade’ template

userlist.js
var MongoClient = mongodb.MongoClient;
var db= MongoClient.connect(mongo.url, function(err, db) {
if(err) {

  1. log(“Failed to connect to the database”);

} else {

  1. log(“Connected to database”);

}
var collection = db.collection(‘phonebook’);
//Get all records and display them

  1. find().toArray(function(err, items) {
  2.    log(items);
  3. render(‘userlist’, {

“userlist” : items
});
});
});

  1. jade

This template displays the list of users as a table. The code is shown below

extends layout
block content
h1= "Display the list of Users"
p
strong Firstname Lastname   Mobile
table
each user, i in userlist
tr
td #{user.FirstName}
td #{user.LastName}
td #{user.Mobile}
p
p
a(href='/') Home

Note: A link back to the Home page is included in here at the bottom.

7

 

5) Adding a User
There are 2 parts to this
a) Invoking the /newuser route to display the input form through the newuser.jade
b) Invoking the /adduser route to insert the values entered in the form. The changes are shown below
a) app.js
..
newuser = require('./routes/newuser')
adduser = require('./routes/adduser')
..
app.get('/newuser', newuser.list);
app.post('/adduser',adduser.list);

b) newuser.js
exports.list = function(req, res){

  1. render(‘newuser’, { title: ‘Add User’});

};

The newuser jade displays the input form
c) newuser.jade
extends layout
block content
h1= "Add a User"
form#formAddUser(name="adduser",method="post",action="/adduser")
input#inputUserFirstName(type="text", placeholder="firstname", name="firstname")
input#inputUserLastName(type="text", placeholder="lastname", name="lastname")
input#inputUserLastName(type="text", placeholder="mobile", name="mobile")
button#btnSubmit(type="submit") submit
p
p
a(href='/') Home

3

d) adduser.js

The adduser.js gets the mongo url from the process.env.VCAP_SERVICES and  setups up the connection to the DB and inserts the values received in the ‘newuser.jade’ form into the database

exports.list = function(req, res) {
if (process.env.VCAP_SERVICES) {
var env = JSON.parse(process.env.VCAP_SERVICES);
if (env['mongodb-2.2']) {
var mongo = env['mongodb-2.2'][0]['credentials'];
}
} else {
var mongo = {
"username" : "user1",
"password" : "secret",
"url" : "mongodb://user1:secret@localhost:27017/test"
}
}
// Set up the DB connection
var MongoClient = mongodb.MongoClient;
var db= MongoClient.connect(mongo.url, function(err, db) {
if(err) {

  1. log(“Failed to connect to the database”);

} else {

  1. log(“Connected to database”);

}
// Get our form values. These rely on the “name” attributes
var FirstName = req.body.firstname;
var LastName = req.body.lastname;
var Mobile = req.body.mobile;
// Set our collection
var collection = db.collection(‘phonebook’);
// Insert the record into the DB

  1. insert({

“FirstName” : FirstName,
“LastName” : LastName,
“Mobile” : Mobile
}, function (err, doc) {
if (err) {
// If it failed, return error

  1. send(“There was a problem adding the information to the database.”);

}
else {
// Redirect to userlist – Display users

  1. location(“userlist”);

// And forward to success page

  1. redirect(“userlist”);

}
});
});

If the insert is successful the userlist page is displayed with the new user

4

6) Updating a User & Deleting a User: Updating and Deleting users follow the same format as Adding a user.

7) index.jade The Home page is built using index.jade with a hyperlink invoking the route for each database operation
extends layout
block content
h1= title
p Welcome to #{title}
ul
li
a(href='/userlist') Display list of users
li
a(href='/newuser') Add a user
li
a(href='/changeuser') Update a user
li
a(href='/remuser') Delete a user

Tip: “Return of the Jadei : Getting the jade template right is truly an art as Jade is extremely finicky about spaces, tabs, indents and outdents(???). Creating the Jade template had me run into circles. I found out that you can debug the jade template individually by executing

C:> npm install jade -g"
and then  running
C:> jade <template name>

from the command prompt. If the result of the command is “rendered <template name>.html” then you are in luck and you can incorporate this jade template into your views folder for e.g.

C: >jade index.jade
rendered index.html

8) Push changes to Bluemix: Once the changes have been made push the changes on to Bluemix with ‘cf’ as follows

cf login -a https://api.ng.bluemix.net
cf push bluemix-mongo -p . -m 512M
cf create-service mongodb 100 mongodb01
cf bind-service bluemix-mongo mongodb01

 

The last 2 commands can also be performed through the Bluemix dashboard in which you add the mongodb service to your Node.js app/

8) Files and Logs: In the Bluemix dashboard you can check your logs in the Files and Logs

5

 

6

Important tip: Finally if the application fails to start when you  push the application with ‘cf’ for e.g.

cf push <app name> -p . -m 512M
....
.....
----> Writing a custom .npmrc to circumvent npm bugs
----> Installing dependencies
----> Caching node_modules directory for future builds
----> Cleaning up node-gyp and npm artifacts
----> No Procfile found; Adding npm start to new Procfile
----> Building runtime environment
----> Checking and configuring service extensions
----> Uploading droplet (7.6M)
of 1 instances running, 1 down
of 1 instances running, 1 down
of 1 instances running, 1 down
of 1 instances running, 1 down

or if  it crashes when you click a link then your debugging friend is

cf logs <app name > — recent
This will dump the error that was encountered either while the application was being started of why the application crashed.

You can fork this Bluemix application from bluemix-mongo at  IBM Devops or from GitHub at bluemix-mongo

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

You may also like
1. Brewing a potion with Bluemix, PostgreSQL & Node.js in the cloud
2. A Bluemix recipe with MongoDB and Node.js
3. A Cloud Medley with IBM’s Bluemix, Cloudant and Node.js
4. Rock N’ Roll with Bluemix, Cloudant & NodeExpress


Find me on Google+

Mixing Twilio with IBM Bluemix


This post walks you through the steps to get started with Twilio on IBM’s Bluemix. Twilio comes as a service that you can add  to your Mobile Cloud or Node.js app. Here’s a quick look at Twilio. Twilio, is a cloud communications IaaS organization which  allows you use standard web languages to build voice, SMS and VOIP applications via a Web API.

Twilio provides the  ability to build VOIP applications using APIs. Twilio itself resides in the cloud and is always available. It also provides SIP integration which means that it can be integrated with Soft switches. Twilio looks really  interesting with its ability to combine the  cloud, Web and VOIP, SMS  and  the like.

This post barely scratches the surface of Twilio & Blue mix. This article provides aa hands-on experience for integration of Twilio with Bluemix and is based on this Twilio blog post. It enables you to send a SMS to your mobile phone by typing in a URL.

As in my earlier post the steps are

1) Fire-up a Node.js  Webstarter application from the  Bluemix dashboard.  In my case I have named the application websms. Once this is up and running

2) Click Add a Service and under ‘Web and Application’ choose Twilio.

3) Enter a  name for the Twilio service. You will also need the Account SID and Authorization token

4) For this go to http://www.twilio.com and sign up2

5) Once you have registered, go to your Dashboard for the Account SID and Auth Token. If the Auth token is encrypted, you can click the ‘lock’ symbol to display the Auth token in plain text.

6) Enter the Accout SID and Auth Token in the Twilio service in Bluemix

7)  To get started you can simply  fork my Twilio  websms code from devops.

8) Now clone the code into a folder you create as follows

git clone https://hub.jazz.net/git/tvganesh/websms

9) You will need to modify the following files

package.json

manifest.yml

app.js

 

10) You can create package.json by running
npm init. Make sure you enter the name of the application you created in Bluemix. In my case it is “websms’ For the rest of the options you can choose the default. Here is the package.json file
"name": "websms",
"version": "0.0.0",
"description": "This README.md file is displayed on your project page. You should edit this \r file to describe your project, including instructions for building and \r running the project, pointers to the license under which you are making the \r project available, and anything else you think would be useful for others to\r know.",
"main": "app.js",
"dependencies": {
"gopher": "^0.0.7",
"express": "^3.12.0",
"twilio": "^1.6.0",
"ejs": "^1.0.0"
},
"devDependencies": {},
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"repository": {
"type": "git",
"url": "https://hub.jazz.net/git/tvganesh/websms"
},
"author": "",
"license": "ISC"
}

11) In the manifest.yml make sure you enter the name of your application and the host

applications:
- host: websms
disk: 1024M
name: websms
command: node app.js
path: .
domain: <your domain>
mem: 128M
instances: 1

12) Lastly make changes to your app.js.

// dependencies
var app = require('gopher'),
twilio = require('twilio');
var config = JSON.parse(process.env.VCAP_SERVICES);
var twilioSid, twilioToken;
config['user-provided'].forEach(function(service) {
if (service.name == 'Twilio') {
twilioSid = service.credentials.accountSID;
twilioToken = service.credentials.authToken;
}
});
// URL test
app.get('/', function(request, response) {
var client = new twilio.RestClient(twilioSid, twilioToken);
client.sendMessage({
to:'<Your mobile number>',
from:'<Number from Twilio dashboard',
body:'Twilio notification through Bluemix!'
}, function(err, message) {
response.send('Message sent! ID: '+message.sid);
});
});

13) After you have made the changes you will need to push the changes to Bluemix using the command line based ‘cf’ tool
14) Login into cf with
cf login – a http://api.ng.bluemix.net

15) Push the websms onto bluemix

16) In the folder where you websms files reside entr the following command
cf push websms -p . -m 512M

17) This should push the code to Bluemix.
Note: If you happen to get a
Server error, status code: 400, error code: 170001, message: Staging error: cannot get instances since staging failed
then you need to make sure to check the changes made to  files app.js, package.,json or the manigfest,yml.

18)  If all things went smoothly, go to your Bluemix dashboard and click the link adjacent to the Routes. You should see that an SMS has been sent as shown

3

19) Your mobile should now display the message that was sent as shown below
Screenshot_2014-06-22-13-41-44

20) Check the  analytics in your Twilio dashboard
1

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

Find me on Google+

Test driving Push notification in Bluemix


This post is a continuation of my earlier post ‘Getting started with mobile cloud in Bluemix‘. Here I take a test drive of the push service that Bluemix offers based on the article “Extend an Android app using the Push cloud service” from developerWorks.

This post assumes that you have already completed the changes from my earlier post for the mobile cloud. If you haven’t,  you could clone the code from “mobile data” which is the official IBM version of this app and includes all the changes needed for persisting data in the cloud through their Android.

The Mobile Cloud App I created on Bluemix is “mobtvg“. The main steps to have Push notification service using Bluemix are

  1. GCM services : Get Google API Project number  & GCM API Key
  2. Include the Google Play services library project
  3. Add the jar files to enable Push service
  4. Modify the server side Node.js file to send push notifications to all registered devices
  5. Make necessary code changes
  6. Run the application and test for notification

Here are more details on the above steps

a) GCM services : Google Cloud Messaging for Android (GCM) is a service that allows you to send data from your server to your users’ Android-powered device, and also to receive messages from devices on the same connection. The 1st thing to do is get the Google API Project number & GCM API key.

– Click on Google Developer Console

– Click Create Project. Enter Project name & click Create.

– Note the Project Number on top of the page.

– Click API & Auth on left panel. Click API.

– Scroll down and turn-on Google Messaging for Android

–  Click credentials and click “Create new key”. Click server key. Click create

-Copy API key in the Public API access

Now go the Bluemix dashboard and click your application. Click the Push module. In the Configuration tab, scroll down to Google Cloud Messaging and  click ‘Edit’

Enter the Google API Project Number & GCM API key for both the Sandbox & Production configuration and click Save.

4
b) In Eclipse click Windows->Android SDK manager. Scroll down to the bottom and under Extras select Google Play services. Click install. Once the installation is successful import the project as follows File-Import->Android->Existing Android  code into Workspace. Click Next. In the next screen Browse to the path where your ADT bundle is installed and choose the folder

<ADT-Bundle>\ sdk\extras\google\google_play_services

and Click Ok. Also  check ‘Copy project into workspace’. This will copy.

3

Now build the Google Play Services Project. To do this the project. Click Project->Properties->Android and make sure that you select ‘Is library project’ and then click build.

5

Add a reference to the Google Play services in the Androidmanifest.xml

<meta-data

android:name=”com.google.android.gms.version”

android:value=”@integer/google_play_services_version” />

c) Make all the code changes given in Step 4 of “Extend an Android app using the Push cloud service.

d) In MainActivity.java make sure you change the app_name to the name of your app for e.g

public static final String CLASS_NAME = “MainActivity”;

public static final String APP_NAME = “mobtvg“;

Also ensure that under assets folder you have populated the Application ID in the bluemix.properties file

applicationID=<Application ID from Bluemix>

d) Add ibmcloudcode.jar, ibmpush.jar, android-support-v4.jar (from <Android_SDK_Location>/extras/android/support/v4)

e) Now the Mobile Push project need to include this library project. To do this select your Mobile App project. Click Project->Properties->Android. Click Add and select google-play-services-lib. Note: Make sure “Is library project” is unchecked otherwise you are in for a lot of grief.

8

f) Now you need to make changes to the Node.js application to push any changes from the server to all registered devices.  The code for this is in bluelist-push-node. Note; Making changes through the GUI results in an error that “manifest.yml is not in root node”. So I suggest that you take the ‘cf’ route as follows.

– Clone the code using Git

git clone https://hub.jazz.net/git/mobilecloud/bluelist-push

Go to bluelist-push-node folder

i) Open the app.js with your favorite editor and enter the Application ID of your Bluemix application

//Data Values

var values = {

version:”0.3.1″,

//change this to the actual application id of your mobile backend starter

appID : “<APPLICATION ID>”,

host : “https://mobile.ng.bluemix.net&#8221;

}

ii) Open manifest.yml and change host name & name to the name of your application for e.g.

host: mobtvg

disk: 1024M

name: mobtvg

command: node app.js

path: .

domain: ng.bluemix.net

mem: 128M

instances: 1

iii) Once the changes are complete, open a command propmpt and  login into Bluemix using ‘cf’ as follows

– cd to the directory in which Node.js & manifest.yml exist, Do

cf login – a http://api.ng.bluemix.net

cf push mobtvg -p . -m 512M

(Note the changes are pushed to the mobile cloud app on Bluemix)

This will run through and finally give the status that the app is running successfully.

f) Now that all changes are complete the Mobile Cloud with Push can be tested..

g) Click Window->Android Virtual Device Manager. Click the Device definitions. You choose Google Nexus, Nexus 7. Click Create AVD.

Note: Make sure you choose Google API Level Y and not Android x.x.x API Y.

6

Let the AVD come up and display the current items in the grocery list.

h) Login to Bluemix. Click Push and select the Notifications tab and enter a test message for e.g. “This is a notification from Bluemix” and click send.

7

This will result in a Push Notification to be sent to the AVD. You should see this popup on you AVD as shown below

1

i) Add another AVD through Windows-Android Virtual Device manager. While one AVD is running go to Run->Run Configurations->Target Device and choose the newly created AVD.

j) This will start a second AVD which will refresh with the contents of the grocery list. Now adda new item in one of AVD devices. This will result in a Push notification to the other device that the Bluelist has been updated.

2

There you have it.

1) A mobile cloud applications in which changes persist in the cloud and are refreshed each time the Android device is restarted.

2) A Push notification that is sent to all registered devices whenever there is a change to the list.

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

Find me on Google+

Getting started with a Mobile Cloud app with Bluemix


This post gives the key steps to get going in building a Mobile Cloud application on IBM’s Bluemix. This post focuses on using the Android Platform for building the application. IBM Bluemix’es mobile cloud application includes under its hood mobile services like mobile application security, push and mobile data. A Node.js is also thrown in to provide server-side functions.

The Bluemix Mobile architecture is shown below

BuildingMobile

 

As in the previous post an existing Mobile cloud application IBM’s bluelist -base is cloned to get familiarity with the steps involved. The IBM’s bluelist-base app enables the user to maintain a grocery list that persists as mobile data in the cloud instance. To get started perform the following

1) Install ADT + Eclipse bundle from the aforementioned link

2) Unzip and install Eclipse and the ADT bundle

3) Make sure you have the Java JDK for Eclipse. If not install from the following site Java SE Development Kit 8 Downloads

4) Since we will be cloning an existing application and using Eclipse to make the changes we need to install EGit.

5) To do this open Eclipse and select Help-> Install New Software and type in http://download.eclipse.org/egit/updates in the Work with text field and hit enter. You should see the following

1

6) Once EGit is installed the IBM’s bluelist-base App can be cloned as follows

7) In Eclipse click File->Import->Git->Import from Git and click Next

8) Choose Clone URI and Click Next

9) Enter the URI for IBM’s bluelist-base. This shown below

2

10) This will download all the necessary source files and other Android related files and directories into the workspace.

11) After this perform the Steps 2 to Step 6 from the link given Build an Android app using the MobileData cloud service

12) After you make the necessary code changes you are good to go

13) Make sure you right-click and add all the necessary imports required (also Ctrl+Shift + O)

14) Build the Project and make sure that there are no errors

15) You are now ready to run the mobile cloud application. We need to run the mobile app on a Virtual simulator. This can be done as

a) In Eclipse click Window->Android Virtual Device Manager. Click the Device Definitions tab.

b) Choose Nexus 7 (Google) and Click Create AVD.
c) This will open a New Window. Set the following Skin->QVGA and Enter 100 MiB in SD Card size and click OK. This will add this as a AVD.

16) Now run the application.

17) This will bring up the AVD. This takes some time You should see the IBM bluelist showing up as one of the apps.

18) Click on IBM Bluelist. You can add grocery items. These items will persist even if you have to restart your application

3

19) The data is persisted in the IBM’s cloud. This can be checked by logging into BlueMix’es dashboard

4

20) Click the Mobile Data and the data entered in the AVD device will show up in Data Classes drop down.

5

21) The Analytics tab will give a graphical output of the API calls

6

So not the mobile app that is cloud enabled is ready.

Clearly the ability to build Android Apps with the data stored at a cloud opens up numerous possibilities for apps like Evernote, Pocket across several devices.

There you have your first Mobile Cloud App.

Watch this space!

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

Find me on Google+