Cognitive Listening

For a decade now, we’ve lived in a “social” era – through networks such as LinkedIn, Facebook, and IBM Connections.  Social networks have given us the tools by which we can engage in dialogue, share ideas, and find new information.  A simple example is the now-ubiquitous comments section seen on most websites.  Someone posts content, and someone writes back in the form of a comment.  Collectively, those comments reflect public perception, understanding, and support.  But to get a sense of the public’s reaction, you have to read through all of the comments.

This is where the emerging cognitive era can help.  Rather than manually read the comments, one could simply use a cognitive service to classify the emotions of commenters.  One such service is IBM’s AlchemyAPI.  Below I’ve combined Alchemy’s emotion analysis with IBM Connections’ comments to generate a “social reaction” to my post.

Connections Sentiment

Admittedly, people aren’t really angry with my post – maybe it’s the exclamation marks being used by commenters … but you get the point.  In isolation, this is a neat trick.  But when you apply this on a larger scale, it gives you the ability to listen cognitively to the social network.  For example, an active forum of actual angry customers could trigger intervention by a customer support representative.  Or combined with other services – like concept extraction – would tell us which areas of the company, initiative, or project that employees are struggling with.  The possibilities and outcomes are substantial, which is why cognitive is more than just technology. It’s a new era of business and computing.

Getting Started

  1. Lots of information exists on using AlchemyAPI.  Start out by creating an account on Bluemix and adding the service.
  2. I used a tool called Greasemonkey to add the “Reaction widget” to IBM Connections Blog pages.  Think of Greasemonkey as a way of creating small, personal applications that run only in your browser.
  3. Adapt my widget below to experiment with content an APIs.
// ==UserScript==
// @name        Blog Entry Emotion Analyzer
// @namespace
// @include     https://apps.**/entry/*
// @include*/entry/*
// @version     1
// @require
// @require
// @grant       GM_xmlhttpRequest
// ==/UserScript==

console.log("Starting up Blog Entry Emotion Analyzer Widget");

// setup the widget in right side column
var sidebar = $( ".lotusColRight" )

  // any html added to DOM MUST USE SINGLE QUOTES
  sidebar.append("<div aria-expanded='true' name='reaction_section_mainpart' class='lotusSection' role='complementary' aria-label='Tone' aria-labelledby='section_reaction_label'><label class='lotusOffScreen' aria-live='polite' id='reaction_section_hint_label'>Expanded section</label><h2 style='cursor:default'><span id='section_authors_label' class='lotusLeft'>Reaction</div></span></h2><div id='section_reaction' class='lotusSectionBody'><span class='lotusBtn lotusLeft'><a id='analyzeButton' role='button' href='javascript:;'>Analyze Comments</a></span><canvas id='watsonChart' width='300' height='300'></canvas></div></div>");
  // attach an event handler to do the analysis when button is clicked
  $("#analyzeButton").click(function() {
    // inform the user something is happening
    $("#analyzeButton").text("Analyzing ...")
    // get the html of the blog entry
    var entryHtml = $("div.entryContentContainer");

    // get the html of the blog comments
    // var commentsHtml = $("#blogCommentPanel"); // Connections Cloud
    var commentsHtml = $( "div[dojoattachpoint='commentsAP']" ); // Connections on-prem

    // decide whether you want to use AlchemyAPI against comments or the entry
    if(commentsHtml.length) {

    } else {
      console.error("Could not find text entry; can't add widget");
} else {
  console.error("No sidebar found in HTML; can't add widget");

function post(html) {
  // any HTML text sent to AlchemyAPI needs encoded
  html = encodeURIComponent(html);
   console.log("Sending text to AlchemyAPI: " + html);
  // send the html to watson APIs
    method: "POST",
    headers: {
      "Content-Type": "application/x-www-form-urlencoded"
    url: "",
    data: "apikey=<use your own API key>&outputMode=json&html=" + html,
    onload: function(response) {

      // received response; construct the widget
    onerror: function(response) {
function createChart(response) {
  console.log("Creating chart");
  // remove the analyze button from the view
  // convert the API response to JSON
  var json = JSON.parse(response.responseText);
  // set up the chart
  var ctx = $("#watsonChart");
  if(ctx == undefined) {
    console.error("Context for chart not found");
  var data = {
    labels: ["Anger", "Disgust", "Fear", "Joy", "Sadness"],
    datasets: [
        label: "Sentiment",
        backgroundColor: [
          'rgba(255, 99, 132, 0.2)',
          'rgba(75, 192, 192, 0.2)',
          'rgba(153, 102, 255, 0.2)',
          'rgba(255, 206, 86, 0.2)',
          'rgba(54, 162, 235, 0.2)'

        borderColor: [
          'rgba(75, 192, 192, 1)',
          'rgba(153, 102, 255, 1)',
          'rgba(255, 206, 86, 1)',
          'rgba(54, 162, 235, 1)'

        borderWidth: 1,
        data: [
  // add the chart to the view
  var myBarChart = new Chart(ctx, {
      type: 'horizontalBar',
      data: data,
      options: {
        title: {
          display: false
        legend: {
          display: false

Installing Greasemonkey Reaction Widget

  1. Launch your Firefox browser.
  2. Head over to the Greasemonkey addon page.
  3. Click the “Add to Firefox” button.
  4. You’ll then see a little monkey on the toolbar.
  5. Copy the script above to the clipboard.
  6. Click “Add New User Script”.
  7. Click “Use Script From Clipboard”.
  8. Change the script as needed.

New User Script

Greasemonkey Script

Greasemonkey Editor

How It Works

A few things to point out:

  • The top of the script defines where the “application” can run.  I’ve made it so that the widget will be added to IBM Connections Cloud and IBM’s Connections deployment.  You should update the @include line to reflect your server installation.  The @include directive also says to run the application only on Blog entry pages.  It does not currently run on a wiki or forum page for example.
  • The script will add a button to the right sidebar.  Pressing the button invokes the AlchemyAPI.
  • The text sent to AlchemyAPI is obtained from the Comments section of the post.  All we’re doing here is grabbing the HTML from inside your browser and making an API call.  AlchemyAPI does the rest.
  • I’m using Chart.js to create the chart.  I’ve used it before on other blog posts.
  • The color of the emotions in the chart is similar to the “Inside Out” characters. 😉

Inside OutHappy coding!

Using cURL with IBM Connections Cloud

I love Java. But there are times that writing a program is more work than it’s worth.  And to the novice, trying to get set up with a JVM, IDE, etc only adds to the time commitment.

So I re-introduce you to cURL (I’ve mentioned it a few times on the blog).  What is cURL?  It’s like a browser – only without the user interface.  cURL gets and sends the raw text data to and from a server.  This is what you see when you use the “View Source” option in your web browser.

I’ll use cURL to populate a bunch of Connections Cloud communities quickly. (You could do this for on-premises as well.)  For example, let’s say my company just moved to Connections Cloud. And for every network shared folder we previously used to be organized (terrible), we’d rather use a Connections Cloud community (awesome).  The reason to leverage cURL to do this is that creating the community is very easy. And it’s something you’ll do once or occasionally.  So a scripted approach is more efficient than writing code.

Let’s get to it.  For reference, review the cURL scripts I have laying around in Just unzip it to any Windows computer.


You can either download cURL or use the one I’ve packaged in my  I’d recommend using mine since it works with the rest of the examples.


Every cURL script I create starts with some setup to initialize parameters like server URL, username, and password.  The first time you run the scripts, it will prompt for user name and password.  Anything run subsequently will be done in the context of this user name (e.g. My Communities).


The below command sets the path to the cURL executable.  It also ensures that basic authentication is used and the username:password pair are included any time a Connections Cloud script is run.

set curl=%~dp0/curl/curl.exe -L -k -u %cnx_username%:%cnx_password%


This script set the URL to the server.  It also prompts the user for credentials if not already provided previously.

@echo off
REM CA1 Test Server
REM set cnx_url=
REM North America Production Server
set cnx_url=
IF DEFINED cnx_url (echo %cnx_url%) ELSE (set /p cnx_url= Connections URL:)
IF DEFINED cnx_username (echo %cnx_username%) ELSE (set /p cnx_username= Connections ID:)
IF DEFINED cnx_password (echo **masked**) ELSE (set /p cnx_password= Connections Password:)


Next we need to create a community.  This is done simply by sending text to the Connections Cloud server.

The Script

The cURL script looks like the following.

@echo off
call ../SetupCnx.bat
call ../SetupCurl.bat
%curl% -v -X POST --data-binary @%1 -H "Content-Type: application/atom+xml" %cnx_url%/communities/service/atom/communities/my

A couple of points:

  • -v is the verbose flag; I use it to see everything that happens. You can remove it if you’d like
  • –data-binary @%1 means that I am sending a file to the server and the file name is provided as input on the command line
  • -H “Content-Type: application/atom+xml” is a required setting; you need to set a header specifying the content type per the API doc
  • %cnx_url%/communities/service/atom/communities/my is the URL to the Connections endpoint per the API doc

To create the community, all that’s needed is to create an XML file and run the following command.

C:\IBM\workspaces\connections\cURL\communities>CreateCommunity.bat CommunityInpu

The Input

The above command has CommunityInput.xml at the end.  This is the input file that is used to create the community. The input XML file is easy on the eyes as well.  If we had multiple communities, I would write a few more lines in the script to substitute the list of folders for the title field.  Or you could create more input files … it’s a lot easier to edit text than program.

<?xml version="1.0" encoding="UTF-8"?>
<entry xmlns="" xmlns:app=""
 <title type="text">Community Name Goes Here</title>
 <content type="html">Community Description Goes Here</content>
 <name>Van Staub</name>
 <name>Van Staub</name>
 <category term="community" scheme=""></category>

I’ve boldfaced the areas you might want to change. But use the API doc as a guide of what you can additionally set.  Most importantly the snx:userid applies to either your GUID for Connections on-premises or your subscriber ID for Connections Cloud.

That’s it.

  1. Unzip my sample.
  2. Update the CommunityInput.xml.
  3. Run CreateCommunity.bat

So next time you need to get something completed quickly or just want to experiment with the APIs, take a look at the cURL scripts I posted.  Most of them should work …

Happy scripting!

Building Social Applications using Connections Cloud and WebSphere Portal: SAML and Single Sign-On

If you’ve been following along, we’ve created a custom solution that combines WebSphere Portal and Connection Cloud to create a socially enabled web site.  To access information in Connections Cloud, OAuth is used as the mechanism to exchange data.  Unfortunately, OAuth does not do everything.  If a user were to follow a link from Portal to Connections Cloud, he or she would need to log in to Connections Cloud.  What gives?

The reason is that WebSphere Portal is what authenticates to get the user’s data, but the actual user’s browser is not authenticated.  The result feels like a disconnect for users and non-technical observers.  The solution to this problem is SAML or Security Assertion Markup Language.  I’ve written about SAML on this blog previously, see Using SAML with SmartCloud for Social Business.  What I’ll do here is add to that work to create a solution that uses WebSphere Portal.


The general flow goes a bit like this:

  1. “Something” triggers the SAML process.  It could be by a strict precondition (like right after you login) or dynamically (something realizes that you have not yet authenticated).
  2. Send the user to a web application hosted on Portal.  This web application (the actual page a user visits) is designed to construct the SAML token.
    1. The SAML token is signed using a certificate previously exchanged with IBM.  There is a manual Support process that you must follow for this to work.
  3. The web page then POSTs the SAML assertion to Connections Cloud.
    1. Connections Cloud decrypts the token, inspects the user’s identity and allows access if appropriate.


Why would I ever do this?!?!  There are existing solutions I could use: Tivoli Federated Identity, Microsoft Active Directory Federation Server, Shibboleth.  But I needed something narrow in scope.  I don’t want an identity server – I already have one, Portal.  And I don’t want to add more servers to the existing deployment.  So I’ve created a module that does only one thing: determines who you are from Portal, creates a SAML assertion, and sends it to Connections Cloud.

That and I had the code laying around … it just needed a use case.


This is a fairly technical project.  Even my eyes glaze over when I starting hearing about ciphers and key chains, but the main moving parts are as follows.

SAML Servlet

The SAML Servlet listens for incoming requests.  In doing so, it will do the following:

  1. Figure out the user’s identity.  The web module is protected and thus all users must be authenticated by WebShere to access.
  2. Construct the SAML token.
  3. Generate a web page with the form that sends the token to Connections Cloud.  (Javascript will submit the form automatically for the user.)
    1. There’s also a bit of code that will realize the rediret (302) coming from Connections Cloud if the admin has configured to use only this servlet as the identity server.
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
public class Saml11Servlet extends HttpServlet {
private static final long serialVersionUID = 1L;
private Saml11ServletConfig config;
private ISamlSigner signer;
private SamlCreator creator;
private ISamlIdentityProvider idProvider;
private final String emailParam = "email.domain";
public void init() throws ServletException {
config = new Saml11ServletConfig(this.getServletConfig());
signer = new SimpleSamlSigner(config.getKeyStorePath(),
config.getKeyStorePassword(), config.getKeyStoreAlias());
creator = new SimpleSaml11Creator(config, signer);
// see
// for additional ways to get the email from a logged in user
// be careful as not all WebSphere servers use VMM (i.e. Federated Security)
idProvider = new PrincipaltoEmailIdentityProvider(this.getServletConfig().getInitParameter(emailParam));
protected void doGet(HttpServletRequest request,
HttpServletResponse response) throws ServletException, IOException {
// incoming 302 from Connections Cloud has TARGET parameter
String target = request.getParameter("TARGET");
String identity = idProvider.getUserIdentity(request.getUserPrincipal().getName());
String token = creator.create(identity, null);
response.getWriter().print(getForm(config.getEndpoint(), token, target));
private String getForm(String endpoint, String token, String target) {
return "&lt;html xml:lang=\"en\" xmlns=\"\"&gt;&lt;head&gt;&lt;meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\"&gt; &lt;title&gt;SAML POST response&lt;/title&gt; &lt;/head&gt; &lt;body&gt; &lt;form method=\"post\" action=\"" + endpoint + "\"&gt;&lt;p&gt;&lt;input name=\"TARGET\" value=\"" + target + "\" type=\"hidden\"&gt; &lt;input name=\"SAMLResponse\" value=\"" + token + "\" type=\"hidden\"&gt; &lt;noscript&gt; &lt;button type=\"submit\"&gt;Sign On&lt;/button&gt; &lt;!-- included for requestors that do not support javascript --&gt; &lt;/noscript&gt; &lt;/p&gt; &lt;/form&gt; &lt;script type=\"text/javascript\"&gt; setTimeout('document.forms[0].submit()', 0); &lt;/script&gt; Please wait, signing on... &lt;/body&gt;&lt;/html&gt;";

Identity Provider

This is pretty basic.  I’m assuming that the user currently has a Portal session.  If so, I’m grabbing the Principal (e.g. wpadmin) and then appending a configurable email domain.  The email address is a required format for Connections Cloud.  Thus you can implement your identity lookup any way you want, but the final value must be an email address that is also stored in Connections Cloud.

public class PrincipaltoEmailIdentityProvider implements ISamlIdentityProvider {
	private final String domain;
	public PrincipaltoEmailIdentityProvider(String domain){
		this.domain = domain;
	public String getUserIdentity(String userId) {
		return userId + "@" + domain;

SAML Token Creator

Things are about to get interesting.  With the user’s identity, we need to construct the SAML token.  I’ve chosen a SAML 1.1 implementation … because it was easier.  This code simply takes the template XML and substitutes appropriate values.  The result at the end is really just XML.  But this is where mistakes happen.  The values used must be accurate in not only the value but also format (e.g. date).

import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.Date;
import java.util.Iterator;
import java.util.Map;
import java.util.Map.Entry;
import java.util.SimpleTimeZone;
import org.apache.commons.lang.StringEscapeUtils;
public class SimpleSaml11Creator extends SamlCreator {
	private static final String SAML_11_TEMPLATE = "&lt;samlp:StatusCode " + "Value=\"samlp:Success\" /&gt;$AUDIENCE$"
			+ "&lt;saml:AuthenticationStatement " + "AuthenticationInstant=\"$AUTH_INSTANT$\" AuthenticationMethod=\"urn:oasis:names:tc:SAML:1.0:am:password\"&gt;"
			+ ""
			+ "$USER_NAME$"
			+ "urn:oasis:names:tc:SAML:1.0:cm:bearer"
			+ "&lt;saml:NameIdentifier " + "Format=\"urn:oasis:names:tc:SAML:1.0:assertion#emailAddress\"&gt;$USER_NAME$"
			+ "urn:oasis:names:tc:SAML:1.0:cm:bearer"
			+ "$ATTRIBUTES$";
	public static final String ATTRIBUTE_FORMAT = "$ATTR_VALUES$";
	public static final String ATTRIBUTE_VALUE_FORMAT = "$ATTR_VALUE$";
	public SimpleSaml11Creator(ISamlConfig config, ISamlSigner signer) {
		super(config, signer);
	protected String getToken(String userId, Map&lt;String, String[]&gt; userAttrs) {
		Date now = new Date();
		SimpleDateFormat format = new SimpleDateFormat(DATE_FORMAT);
		Calendar cal = Calendar.getInstance(new SimpleTimeZone(0, "GMT"));
		long validLength = (long) 60000 * config.getTokenExpiration();
		// Setup the time for which SAML assertion is valid
		Date notBefore = new Date(now.getTime() - validLength);
		Date notAfter = new Date(now.getTime() + validLength);
		String saml = SAML_11_TEMPLATE.replace("$ISSUE_INSTANT$",
		saml = saml.replace("$RESPONSE_ID$", now.getTime() + "");
		saml = saml.replace("$AUDIENCE$", config.getEndpoint());
		saml = saml.replace("$RECIPIENT$", config.getEndpoint());
		saml = saml.replace("$ISSUER$", config.getIssuer());
		saml = saml.replace("$ASSERTION_ID$", now.getTime() + "");
		saml = saml.replace("$NOT_BEFORE$", format.format(notBefore));
		saml = saml.replace("$NOT_AFTER$", format.format(notAfter));
		saml = saml.replace("$AUTH_INSTANT$", format.format(now));
		saml = saml.replace("$USER_NAME$", StringEscapeUtils.escapeXml(userId));
		StringBuilder allAttrs = new StringBuilder();
		if (userAttrs != null) {
			Iterator&lt;Entry&lt;String, String[]&gt;&gt; iterator = userAttrs.entrySet()
			while (iterator.hasNext()) {
				Entry&lt;String, String[]&gt; entry =;
				boolean attHasValue = false;
				// Setup the attribute values
				String attrValues = "";
				String[] values = entry.getValue();
				// Make sure values is not null
				if (values != null) {
					for (int j = 0; j &lt; values.length; j++) { String value = values[j]; // Only add the attribute if there is a value to add if (value != null &amp;&amp; value.length() &gt; 0) {
							attHasValue = true;
							attrValues += ATTRIBUTE_VALUE_FORMAT.replace(
				if (attHasValue) {
					// Setup the attribute name and namespace
					String key = entry.getKey();
					String attribute = ATTRIBUTE_FORMAT.replace("$ATTR_NAME$",
					attribute = attribute.replace("$ATTR_NAMESPACE$",
					attribute = attribute.replace("$ATTR_VALUES$", attrValues);
		saml = saml.replace("$ATTRIBUTES$", allAttrs.toString());
		return saml;

SAML Signer

Now that there’s an XML SAML token, we need to sign it.  The signer class doesn’t do the actual signing; I’ve taken care of that implementation in the SAML Creator class.  The signer class’s job is to produce the X509Certificate.  I’ve chosen to simply pull it off disk from a configurable location.  A better implementation is to get it from built-in WebSphere keystores.  And since it’s not that interesting, I’ve left it out of the blog post (though it’s in the project code).

SAML Creator

And now we need to bring it all together. Take the XML, sign it with the certificate and hand it back to the servlet for posting to Connections Cloud.

import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.logging.Logger;
import javax.xml.crypto.dsig.CanonicalizationMethod;
import javax.xml.crypto.dsig.DigestMethod;
import javax.xml.crypto.dsig.Reference;
import javax.xml.crypto.dsig.SignatureMethod;
import javax.xml.crypto.dsig.SignedInfo;
import javax.xml.crypto.dsig.Transform;
import javax.xml.crypto.dsig.XMLSignature;
import javax.xml.crypto.dsig.XMLSignatureFactory;
import javax.xml.crypto.dsig.dom.DOMSignContext;
import javax.xml.crypto.dsig.keyinfo.KeyInfo;
import javax.xml.crypto.dsig.keyinfo.KeyInfoFactory;
import javax.xml.crypto.dsig.keyinfo.X509Data;
import javax.xml.crypto.dsig.spec.C14NMethodParameterSpec;
import javax.xml.crypto.dsig.spec.TransformParameterSpec;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.NodeList;
public abstract class SamlCreator {
	private final Logger logger = Logger.getLogger(SamlCreator.class.getName());
	public static final String DATE_FORMAT = "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'";
	protected ISamlConfig config;
	protected ISamlSigner signer;
	protected abstract String getToken(String userId,
			Map&lt;String, String[]&gt; userAttrs);
	public SamlCreator(ISamlConfig config, ISamlSigner signer) {
		this.config = config;
		this.signer = signer;
	public String create(String userId, Map&lt;String, String[]&gt; userAttrs) {
		logger.fine("Creating SAML token for " + userId);
		String saml = getToken(userId, userAttrs);
		logger.finest("SAML token = " + saml);
		Document doc;
		try {
			DocumentBuilderFactory dbfac = DocumentBuilderFactory.newInstance();
			DocumentBuilder docBuilder = dbfac.newDocumentBuilder();
			ByteArrayInputStream is = new ByteArrayInputStream(
			doc = docBuilder.parse(is);
			logger.finest("Successfully converted SAML token to " + doc.getClass().getName());
			NodeList nodes = doc.getDocumentElement().getChildNodes();
			for (int i = 0; i &lt; nodes.getLength(); i++) {
				// Sign the SAML assertion
				if (nodes.item(i).getNodeName().equals("saml:Assertion")) {
					if (config.signAssertion()) {
						logger.fine("Signing SAML assertion element");
						signSAML((Element) nodes.item(i), null, "AssertionID");
					} else {
						logger.fine("Skipping signing of SAML assertion element");
			logger.fine("Signing SAML response");
			// Sign the entire SAML response
			Element responseElement = doc.getDocumentElement();
					(Element) responseElement.getFirstChild(), "ResponseID");
			// Transform the newly signed document into a string for encoding
			TransformerFactory fac = TransformerFactory.newInstance();
			Transformer transformer = fac.newTransformer();
			StringWriter writer = new StringWriter();
			transformer.transform(new DOMSource(doc), new StreamResult(writer));
			String samlResponse = writer.toString();
			logger.finest("SAML token = " + samlResponse);
			logger.fine("Base64 encoding SAML token");
			// Encode the string and return the response
			return new String(Base64.encode(samlResponse.getBytes("UTF-8")));
		} catch (Exception e) {
		return null;
	private void signSAML(Element element, Element sibling, String referenceID) {
		logger.fine("Signing element " + referenceID);
		try {
// this needs to be here due to Java bug in 1.7_25
            element.setIdAttribute(referenceID, true);
			XMLSignatureFactory fac = XMLSignatureFactory.getInstance("DOM");
			DOMSignContext dsc;
			if (sibling == null) {
				dsc = new DOMSignContext(signer.getPrivateKey(), element);
			} else {
				dsc = new DOMSignContext(signer.getPrivateKey(), element,
			DigestMethod digestMeth = fac.newDigestMethod(DigestMethod.SHA1,
			Transform transform = fac.newTransform(Transform.ENVELOPED,
					(TransformParameterSpec) null);
			List list = Collections.singletonList(transform);
			String refURI = "#" + element.getAttribute(referenceID);
			Reference ref = fac.newReference(refURI, digestMeth, list, null,
			CanonicalizationMethod canMeth = fac.newCanonicalizationMethod(
					(C14NMethodParameterSpec) null);
			List refList = Collections.singletonList(ref);
			SignatureMethod sigMeth = fac.newSignatureMethod(
					SignatureMethod.RSA_SHA1, null);
			SignedInfo si = fac.newSignedInfo(canMeth, sigMeth, refList);
			KeyInfoFactory kif = fac.getKeyInfoFactory();
			List x509Content = new ArrayList();
			X509Data xd = kif.newX509Data(x509Content);
			KeyInfo ki = kif.newKeyInfo(Collections.singletonList(xd));
			XMLSignature signature = fac.newXMLSignature(si, ki);
			logger.fine("Successfully signed element" + referenceID);
		} catch (Exception e) {
			// TODO: throw instead of catch


If all goes well, the result should look something like this.

Download Video: MP4


And you’ll probably need the project code.

This code is as-is for education purposes.  FWIW I’m not a SAML expert; so if you post a question, there’s a good chance I won’t know.

Happy coding.




Building Social Applications using Connections Cloud and WebSphere Portal: Social Portal Pages

We’re going to use Portal’s theme framework to add the necessary CSS and JS files to our social pages.  Using this approach, we’ll no longer need to include the dependencies in our script portlets.  Pages that have social script portlets on them can simply have the relevant theme profile applied.  Another benefit is that by using Portal’s profile feature, the various browser requests are centralized into a single download to reduce the time taken to load the page.

Creating the Theme Modules

Let’s begin by adding new theme modules.  The modules will include the following resources on the page:

  • The Social Business Toolkit SDK’s Javascript dependency, for example /sbt.sample.web/library?lib=dojo&ver=1.8.0&env=smartcloudEnvironment
  • CSS files from Connections Cloud, for example /connections/resources/web/_style?

You can read how to create the module framework in the Knowledge Center.  Since the CSS files are located on a remote server, I need to create a “system” module.  This is essentially creating a plugin with the relevant extensions.  It’s a web project (WAR) with a single plugin.xml file.  The contents of my plugin.xml are as follows.

<?xml version="1.0" encoding="UTF-8"?>
<?eclipse version="3.4"?>
<plugin id=""
name="Social Business Toolkit Theme Modules"
value="Social Business Toolkit SDK">
value="Social Business Toolkit SDK">
value="{rep=WP CommonComponentConfigService;key=sbt.sdk.url}/sbt.sample.web/library?lib=dojo&amp;ver=1.8.0&amp;env=smartcloudEnvironment">
value="{rep=WP CommonComponentConfigService;}/connections/resources/web/_style?">
value="{rep=WP CommonComponentConfigService;}/connections/resources/web/_style?">
value="{rep=WP CommonComponentConfigService;}/connections/resources/web/_lconntheme/default.css?version=oneui3&amp;rtl=false">
value="{rep=WP CommonComponentConfigService;}/connections/resources/web/_lconnappstyles/default/search.css?version=oneui3&amp;rtl=false">

You could use the actual server’s path, for example<some css resource> in the XML. But I’m using a substitution rule

{rep=WP CommonComponentConfigService;}

that will swap the corresponding keys in the plugin.xml for the values defined by WebSphere’s Resource Environment Provider.  The only reason I did this was so I could configure the URLs from WebSphere rather than hard code them into the plugin.xml.


The other thing I’m doing is telling the SBT SDK which environment I want configured by referencing sbt.sample.web/library?lib=dojo&amp;ver=1.8.0&amp;env=smartcloudEnvironment.  This alleviates me from having to manually specify the endpoint in the SBT scripts I write later.  And notice the ampersand symbol amp semicolon format.  You’ll need to escape the ampersands in the plugin.xml.

Create your web module and deploy to your server.  You can use the Theme Analyzer tools in Portal’s administration interface to pick up the new modules.  Just go to the Control Center feature and invalidate the cache.

Invalidate Theme

Then review the system modules to locate the sbt_sdk one.

sbtSdk Module


To actually use the module, we need to build a theme profile.  A profile is a recipe of which modules should be loaded for a particular page’s functionality.  In addition to the sbtSdk module, we’ll need other IBM provided or custom modules loaded for pages to properly work.  Profile creation is rather straightforward.  You can use the existing profiles as a starting point.  See those in webdav for example; I use AnyClient to connect to my Portal  Once there, you can peruse the profiles under the default theme.

I’ve created a SBT Profile that includes the SDK and Cloud modules I created earlier.

 "moduleIDs": ["getting_started_module",
 "deferredModuleIDs": ["wp_toolbar_host_edit",
 "titles": [{
 "value": "Connections Cloud",
 "lang": "en"
 "descriptions": [{
 "value": "This profile has modules necessary for viewing pages that contain portlets written with the Social Business Toolkit SDK and Connections Cloud banner integration",
 "lang": "en"

This JSON file is then added to my default Portal theme using a webdav client.

SBT Profile WebDav

You’ll likely need to again invalidate the theme cache for the profile to be available for the next section.

Page Properties

To enable the profile on a page, we need to update the page properties.  The result of this process is that the aforementioned Javascript and CSS files get added to any page that has the profile enabled.

SBT Profile

And that’s it.  Now any developer can begin authoring “social” script portlets with nothing more than the page profile and a bit of web code.




Building Social Applications using Connections Cloud and WebSphere Portal: SBTSDK

To build social applications, I strongly suggest using the Social Business Toolkit SDK.  While Connections Cloud has simple to use REST APIs, the SDK provides a ready to go foundation.  Admittedly, the SDK is large and can be confusing at first.  In it you’ll find code for a variety of technology platforms: Domino, iOS, PHP, Java, Javascript.  We, developers, tend to want to jump right in and start code.  Don’t.  If you don’t start with the SDK, you’ll find yourself building authentication mechanisms, HTTP clients, XML readers, etc … stuff you really don’t need to re-invent.

There are two ways to get the Social Business Toolkit SDK:

  • Download the SDK and sample web applications from OpenNTF.   Since the SDK and samples are pre-packaged, this allows you to simply install and get going.
  • Download the SDK source and build the projects from GitHub.  While this takes more time, the code is more recent.  The last package posted on OpenNTF is over a year old at the time of writing.

Downloading the Social Business Toolkit SDK from OpenNTF

To download and install, see documentation.  High level steps are:

  1. Download the latest version of the SDK.
  2. Extract the zip and locate the file \samples\ear\sbt.sample- (or similar name).

Building the Social Business Toolkit SDK from GitHub

The documentation on building the source is a bit dated.  Most developers will be able to follow the steps below.

  1. Download the source (e.g. download ZIP) from the SocialSDK repo on SBTSDK GitHub.
  2. Import the maven projects into Eclipse.  The Import -> Maven -> Existing Maven Projects is an option in later versions of Eclipse.  If you do not have this, consider downloading the M2Eclipse plugin.
  3. Depending on how you import, you may need to update the Project Explorer view to include the working set.Maven SBT Working Set
  4. Select the sbt.sample project -> Export -> EAR File.

Installing the Social Business Toolkit SDK Sample

  1. Install the EAR (either the one from Downloading Step 2 or Building Step4) using WAS admin console.  I’m installing the SDK directly to the Portal server.
  2. Verify the application by visiting http://<portal>:<port>/sbt.sample.web/ in a web browser.  (The default port for Portal is 10039.)

SBT Home


Congratulations, now you can get started.

Building Social Applications using Connections Cloud and WebSphere Portal: Your First Social Portlet

Part of the series Building Social Applications using Connections Cloud and WebSphere Portal.

Create a Sample Application

Let’s start simple.  We’ll re-use on the SBT SDK’s sample applications inside Portal.

  1. Go to the SBT SDK application http://<portal>:<port>/sbt.sample.web/javascript.jsp.
  2. Authenticate using OAuth if you did not do so previously.  Do this by clicking the “Login” button on the Authentication -> Authentication Summary sample.
  3. Next select the Social -> Files -> Get My Files sample.
  4. If an error occurs, update line 18 to include the smartcloudOA2 endpoint in the service.
    var fileService = new FileService({endpoint: "smartcloudOA2"});
  5. Click the “Run” button.  You should see a list of files.  (If not, just make sure there are actually files in the My Files area of Connections Cloud for this user.)
  6. Keep the sample open, we’ll use it in Portal.

SBT My Files Sample


Adding the Sample to the Script Portlet

Now we’ll take the same code seen in the Javascript tab and add that to the Script Portlet.  The effect is exact same as the Get My Files sample – only that it’s coming from Portal.

  1. Log in as an administrator or a user with edit rights to Portal pages.
  2. Create a new page in Portal.
  3. Add the Script Portlet to the page from the content palette.  (If you get an error when doing this, confirm the site mapping for the page in the steps below.)
    1. In the Page Properties, select Web Content -> Edit.
    2. Click “Add Web Content”.
    3. Navigate to Libraries -> Script Portlet Library -> Script Portlet Applications.
    4. “OK” and re-add the Script Portlet to the page.
  4. Click “Edit” in the Script Portlet.
  5. Fill in the HTML and Javascript tabs in the Script Portlet with the respective tabs in the SBT SDK sample.Script Portlet JS
  6. Open a new browser tab and navigate to http://<portal>:<port>/sbt.sample.web/library.  You will receive Javascript as a response.  Add this Javascript to the beginning of the Javascript in the Script Portlet.
  7. Save the Script Portlet.
  8. Exit “Edit Mode” to view the page.

You should now see the same list of files that you saw in the SBT SDK sample on your Portal page.  It may not be pretty, but it is functional.  Try copying other examples from the SDK sample application to Script Portlets.

Get My Files Portal

Some may be wondering why I needed to copy the Javascript located at http://<portal>:<port>/sbt.sample.web/library into the Script Portlet.  This is required to:

  • Ensure the endpoint smartcloudOA2 is available
  • Set the AMD paths to the SDK’s modules.  If this was not done, we’d see the SDK trying to load modules from a Portal context.  Rather they must be loaded from the SDK enterprise application.

Copying this directly into the Script Portlet isn’t ideal.  What if we have two Script Portlets, do we copy into both?  One solution is to include this requirement as an external Javascript resource by adding the http://<portal>:<port>/sbt.sample.web/library URL to the list of dependencies.

SBT Dependency

Another solution I’ve found is use Portal’s page profiles to specify when this bit of code needs to load.  Thus when a Script Portlet containing the SBT SDK exists on the page, I change the page’s profile such that it loads this required Javascript.  I’ll save that discussion for a more advanced post.



Building Social Applications using Connections Cloud and WebSphere Portal

In this series, I’ll explore creating social applications using WebSphere Portal and Connections Cloud.  Specifically, I’ll focus on leveraging the following IBM products and resources:

  • WebSphere Portal (or IBM Web Content Manager)
  • Connections Cloud
  • the Social Business Toolkit SDK
  • WebSphere Portal’s new Script Portlet

To be clear, there are various ways to create a social application in Portal.  Consider reviewing the Redbook Building and Implementing a Social Portal for other options.  If you’d like an IBM off-the-shelf solution, read the Redbook.  But if you’d like to see an alternative approach, follow along:

  1. Downloading and Building the Social Business Toolkit SDK
  2. Getting Started
  3. Your First Social Portlet
  4. Social Portal Pages
  5. Single Sign-On with SAML
  6. Chat-as-a-Service (Coming Soon)

Building Social Applications using Connections Cloud and WebSphere Portal: Getting Started

Part of the series Building Social Applications using Connections Cloud and WebSphere Portal.

Installing the Script Portlet

We’ll be using WebSphere Portal’s Script Portlet.  The reason is that the Script Portlet is ideal for small, web-centric applications.  Much of the Social Business Tookit’s examples are exactly that – small, web-centric apps.  So it will be easy to use the Script Portlat and Toolkit as a starter for social applications.  Also by using the Script Portlet, the technical barrier to creating these applications is lower – assuming you don’t live and breath J2EE portlet development.

If you do not already have the Script Portlet installed, see documentation.  High level steps are:

  1. Download the Script Portlet from the Greenhouse Catalog.
  2. Unpack the downloaded zip and move the scriptportlet-app-1.3.0.paa file to the Portal server.
  3. Run
    ./ install-paa -DPAALocation=/path/scriptportlet-app-1.3.0.paa -DWasPassword=password -DPortalAdminPwd=password
  4. Run
    ./ deploy-paa -DappName=scriptportlet-app -DWasPassword=password -DPortalAdminPwd=password
  5. Restart Portal.

Configuring Connections Cloud

The Toolkit will use OAuth to communicate with Connections Cloud and retrieve data.  To do that, you’ll need to add an “Internal App” in Connections Cloud.

  1. Log in as an administrator or app developer to Connections Cloud (usually
  2. Click Internal Apps -> Register App.
  3. Provide a name and select the OAuth 2.0 radio button.
  4. Set the callback URL to the server where you installed the Toolkit.  For example, my server is
  5. Click Register.
  6. Back on the Internal Apps page, select the drop down for the app you created.
  7. Click “Show Credentials” and click the “Show Client Secret” link.
  8. Leave this screen open; the details will be used next.

OAuth2.0 Settings

Configuring the Social Business Toolkit SDK

Previously, we installed the SBT SDK.  Now it must be configured to work with the Internal App we created.

  1. Using a text editor, create the file
  2. Copy the following into the file.  You will need to update the section “SmartCloud OAuth 2.0 Endpoint Parameters” with settings from the “Show Credentials” screen on the “Internal Apps” page.
    1. # IBM Social Business Toolkit Configuration
      # Library Servlet Configuration
      # SmartCloud OAuth 2.0 Endpoint Parameters
  3. Since this tutorial focuses on Connections Cloud and OAuth, we’ll remove the SDK’s default environments and additional authentication options for clarity.
    1. Create a file called managed-beans.xml with any text editor.
    2. Copy the following into the file.
      <?xml version="1.0"?>
      <!-- Credential store physical implementation -->
      <!-- Default Environment -->
      <!-- SmartCloud OAuth 2.0 -->
      <!-- Endpoint URL -->
      <!-- OAuth parameters -->
      <!-- Trust the connection -->
      <!-- Access to the credential store -->
    3. Save.
  4. Copy the and managed-beans.xml file to the server.  For example, /opt/IBM/WebSphere/wp_profile/sbt.  (I created the sbt directory.)
  5. Create two new JNDI URLs in WebSphere.
    1. Access the WAS admin console.
    2. Navigate to Resources -> URL -> URLs.
    3. Select the scope to be your cell.
    4. Select New to create a URL with the following properties:
      1. Name=SBT Properties
      2. JNDI name=url/ibmsbt-sbtproperties
      3. Specification=file:///opt/IBM/WebSphere/wp_profile/sbt/
    5. Select New to create another URL with the following properties:
      1. Name=SBT Managed Beans
      2. JNDI name=url/ibmsbt-sbtproperties
      3. Specification=file:///opt/IBM/WebSphere/wp_profile/sbt/managed-beans.xml
    6. Save.
  6. Restart the “Social Business Toolkit Sample Enterprise Application” under Enterprise Applications in WAS.

Test the Social Business Toolkit SDK

Time to test whether the above steps worked.  To do that:

  1. Go to http://<portal>:<port>/sbt.sample.web/javascript.jsp in a web browser.
  2. Select the Authentication -> Authentication Summary sample.
  3. Click the “Login” button.
  4. You should see a Connections Cloud login screen.
  5. Provide valid credentials and click “Log In”.
  6. The previous “Login” button should change to “Logout”

OAuth Authenticated

What’s Next

Next, we’ll create a simple application with the Script Portlet and the Social Business Toolkit SDK examples.

Building Social Applications using Connections Cloud and WebSphere Portal: Your First Widget

IBM Verse Application Development


IBM finally released Verse extension points: business card and view/compose widget integration.  See What’s New October 2016.

Technical details can be found here.


On March 31, IBM released its much anticipated solution to social and email, IBM Verse. And part of my role in IBM is demonstrating how business partners can use technology like IBM Verse to build new and engaging products. The IBM team is still in the process of publishing development material. For now, take a look at the Tech Talk replay “A Deeper Look into IBM Verse“. I’ve also created my own presentation on Verse application development. We are currently giving this presentation as part of a one day workshop in the US. If you’d like to discuss further, feel free to contact me.

Verse Application Development

The presentation is framed as what we have today and what is coming. I’ll talk about what we have today below. But alot is coming, and much of the presentation also discusses how we foster application development in our cloud so that when those Verse APIs are released, you can begin using them quickly.

So what’s available today?

Mailto: “API”

I have quotes around API because application developers might argue that this is not API – though it is one way to immediately integrate Verse.  Mailto is a link in a web page. You’ve no doubt seen the Contact Us link that launches your Notes or Outlook desktop client. Those same web links will work with Verse. Web developers can create links that begin a draft email in Verse and optionally add a recipient, subject, and body. For details, see this post.

Verse Search API

OK, now this one is API. It’s also the same way the Verse web client works. When you click a person, you’ll only see emails from that person. This is called a facet and allows you to filter out all the stuff you don’t care about. So you can concentrate on reviewing emails using:

  • People
  • With Attachments
  • With Links
  • Needing Action
  • Waiting For
  • Date Ranges

Your custom application can filter and receive responses in much the same way. To figure out how (since there’s no public API yet), look at the network tool in Chrome, Firefox, etc. And then just start clicking on the facets to see how the URLs are constructed.

IBM Verse Search API


In this screenshot, I’ve just clicked on the Needs Action facet. And I’m given an email from Amy that I’ve flagged as needing my follow up. The URL requests denoted by the lower red arrows show the API calls being made. Below is the actual GET request.*%20OR%20followupstatus%3A%3F)%20AND%20!folderunid%3A%22FFFFFFFF000000000000000000000001%22%20AND%20!folderunid%3A%22FFFFFFFF000000000000000000000002%22))%20(softdeletion%3A0))%20OR%20unid%3A(CED442931844E0E700257E1300089C78)&start=0&rows=31&withunread=1&sort=followupdate%20asc%2Cmaildate%20desc&group=true&group.field=tua0&group.sort=maildate%20desc&group.limit=1&xhr=1

Go ahead and paste the URL in your address bar, you should get back data.  The API endpoint is  And it carries a set of parameters that define the search:

  • wt:json
  • df:body
  • q.op:AND
  • TZ:America/New_York
  • dbfile:livemail
  • q:((((replydate:* OR followupstatus:?) AND !folderunid:”FFFFFFFF000000000000000000000001″ AND !folderunid:”FFFFFFFF000000000000000000000002″)) (softdeletion:0)) OR unid:(CED442931844E0E700257E1300089C78)
  • start:0
  • rows:31
  • withunread:1
  • sort:followupdate asc,maildate desc
  • group:true
  • group.field:tua0
  • group.sort:maildate desc
  • group.limit:1
  • xhr:1

The “q” parameter is the important one. As you click on facets, note how this parameter changes.

And what you’ll get back is a JSON response that you can use in your custom application.  I’ll save explanation of the parameters, request, and response format for the real IBM documentation. But armed with this, you can start investigating and probably get a good idea of how to search and retrieve information from Verse.

What Else?

So what else can you do? I show some of what’s coming in the presentation. But think about the information stored in Verse: your todo list, who’s important to you, email and social content. Because it provides so many possibilities to improve not only email but other applications, this is why Verse really is a #NewWayToWork.


Connections Cloud Navigation Tricks

There are some rather neat navigational tricks that you can use inside IBM Connections Cloud. Unfortunately these tricks are undocumented and as such are likely to be “unsupported” by IBM and could possibly change in the future.  But because I find them so useful, I just have to show you. And hopefully over time, they find their way into product documentation and API.

Top Navigation Bar Extensions

There is a documented way to add new links to the navigation bar in Connections Cloud. This is done through an extension point where you define the text, link, icon, and other properties of the link. By following these instructions, you can add a link to your company’s expense reporting system, or scheduling, or whatever. The problem comes in when you want a menu of links, or rename this default link to something else, or just want something a little more sophisticated.

A Real World Need

So what might be an example of something more sophisticated. Let’s assume you are using communities to group concepts in your company. Some people have communities devoted to clients. Others have communities for specific projects. And if you’re really organized, you have client communities with projects as sub-communities. The problem arises when you go to look at those communities.

Flat Communities

Because communities are listed all together, you lose the semantics. Which ones are my projects and which ones are my clients? So you begin to use tricks like tagging. My client communities are tagged with “client” and projects are “project” tagged. This helps, but you’ll still start with the full list and need to refine based on tags. Not terrible – in fact this is a great start. But we can do better.

Tags, Semantic Glue

If you were to click the tag “client”, you would be looking at all communities that are effectively clients.

Client Communities

The link in your address bar looks like this.

And you can even look at “My Clients” by using a link to communities you are a member (or own) with the tag “client”.

This is neat because we have the semantics of what we want through the links (aided by tags). We just need to go one step farther to provide direct access.

Navigation Tricks

Here’s where it gets really interesting. I want two menus

  1. The first will list “Clients” with a submenu of “My Clients” and “All Clients”
  2. The second follows the same convention but will be for projects

To do this, we need to use a special syntax in the Organizational Extension’s name. The red arrow is the secret sauce. The blue arrows show values that go unused but still need to be populated to save the extension.

Org Extension Trick

We create two of these extensions for both Clients and Projects. The result is two new menus in our navigation bar. Notice that Projects has the submenu with My Projects and All Projects. And clicking My Projects takes you to the communities application that list your communities and then applies the “project” tag. We’re just enforcing the semantics in a dont-make-me-think type way.

New Navigation Menus

So let’s look at that extension trick in more detail. Here is what’s entered in the name field.

@@mod@auth-left.servicesMenu@after@{"id" : "Projects", "topMenu" : "ProjectsMenu", "text" : "Projects", "class" : "feature", "iconUrl": "", "items" : [{ "url" : "", "text" : "My Projects"}, { "url" : "", "text" : "All Projects"}]}

This looks really really esoteric so let’s break it down.

  1. The beginning is the secret syntax to add a new menu at the end of the navigation bar: @@mod@auth-left.servicesMenu@after@{“id” : “Projects”, “topMenu” : “ProjectsMenu”, “text” : “Projects”, “class” : “feature”, “iconUrl”: “”
  2. Next we have a list of items that are the submenu links: “items” : [{ … }]
  3. And each of those submenu links has the URL to the communities application with the tag parameter added: { “url” : “”, “text” : “My Projects”}

That’s all there is to it. The really observant person might also see that there is no more “Communities” link. That’s because I removed it – favoring the “Clients” and “Projects” direct access over generic “Communities”. To remove the communities application, you create another organization extension with the following name .


Final Thoughts

You can use these tactics to remove, rename, and create more intuitive navigation in your Connections Cloud. Or add more tags and facets to build additional semantics. For example to find out what those tags or facets are, use Connections Cloud’s advanced search to find the content you want. Then review the address bar. It will contain the URL with parameters needed in the extension.

Just remember that these changes will apply to everyone in the company. And you’ll want to Ctrl+F5 to clear your browser cache to observe the changes.