DataPower Best Practices

Lately I was trying to find some good documentation on best practices around DataPower.

Because DataPower is very flexible and provide many solution to a single problem, I have experienced that while putting a design around DataPower you can easily get into arguments (not very polite discussion) with other mates. Discussion can go which is best solution in terms of performance, maintenance & deployment.

I found is very hard to reach to one solution when few DataPower developers putting their own heads and trying to reach to a consensus. I then tried to find what IBM recommendation is and found this link.

Hope this link can help

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/DataPower

Compare 2 DataPower Environment (Version 01)

Introduction

FileDiff allows developers/administrators to compare 2 DataPower environments to compare. It’s a java based program which access XML Management Interface of DataPower to compare file system. Output of tool is formatted HTML.

How it works

FileDiff takes a property file as input which has details of DP boxes. The packaged jar run and creates a HTML output.

Download "FileSystemDiff.jar" & Sample Config file "CompareConfig.properties" file.

CompareConfig.properties file contains 2 entries, each for one box. You should have XML management interface privileges. 

#URL DataPower01

dblabel01=box1

dpUrl01=https://host01:5550/service/mgmt/current

domain01=domain

username01=urusername

password01=urpassword

#URL DataPower02

dblabel02=box2

dpUrl02=https://host02:5550/service/mgmt/current

domain02=domain

username02=urusername

password02=urpassword

#Folder

folderToCompare=local:/

Running the tool

> java -jar FileSystemDiff.jar  CompareConfig.properties

The output of this run would generate DiffOutput.html.

Remember the tool always generates output with name DiffOutput.html so rename or move existing report else new report will replace existing one. 

You can email if you need more information related to same. (rohit.goyal6@gmail.com)

Working with JSON in DataPower

To work with JSON type messages, DataPower first converts it to JSONX, a proprietary XML based format to represent JSON message, does transformation and converts back to JSON (using store:///jsonxtojson.xsl) as per the requirements.

To convert to JSONX, DataPower provides 2 different ways

1) Using HTTP Convert Query Action 

2) Setting RequestType as JSON and use "__JSONASJSONX"

Important to notice that both of these methods works on processing rule. Convert Action works as processing action added on processing rule and __JSONASJSONX also work when it's a input for a processing action.

But I have come across requirements where you need JSONX in a XSL file. In that case, a reusable service can be designed using any of above method and can be called from XSL file. Here is how it's going to look like.

DataPower SOMA Interaction using Java

SOMA Interface is one of the good way to interact with DataPower specially when it comes to admin related activities. 

Here is the Simple Java Code to work with SOMA. 

Code Sends Request to DataPower over HTTPS. It opens a connection, send the SOMA request & print the response received from DataPower.

Note: It creates *insecure" HTTPS connection to DataPower.



Sample SOMA Request

<soapenv:Envelope xmlns:man="http://www.datapower.com/schemas/management" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:dp="http://www.datapower.com/schemas/management">

   <soapenv:Header/>
   <soapenv:Body>
      <man:request domain="Developer04">
         <man:get-filestore location="local:"/>
      </man:request>
   </soapenv:Body>
</soapenv:Envelope>

TCP connection test without login to DataPower box

Yes, it's possible to do TCP Connection Test without login into DataPower box. 

In general, DataPower has provided TCP Connection test UI feature on it's WebUI. After login to DataPower box, click on "Troubleshooting" on control panel and you will get option to try TCP Connection Test to any IP and Port combination from DataPower. 

But it's not a very viable solution specially when you have multiple boxes in production. You need some easy way out test TCP Connection without login to DataPower box. 

SOMA Interface can help us here. You can write a SOMA script to request DataPower to confirm TCP connectivity.

This is the Sample of SOMA Script.. 

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:man="http://www.datapower.com/schemas/management">
   <soapenv:Header/>
   <soapenv:Body>
      <man:request domain="DOMAINDEV"> <!-- Replace domain name -->
         <man:do-action>
            <TCPConnectionTest>
               <RemoteHost>xxx.xxx.xxx.xxx</RemoteHost>
               <RemotePort>8081</RemotePort>
            </TCPConnectionTest>
         </man:do-action>
      </man:request>
   </soapenv:Body>

</soapenv:Envelope>

Dynamic Context Variable Set Action

I had a requirement where I need to create and store as many context variable depending on the number of elements received with incoming request message. For example, if incoming request messages is.....................

<in>

   <element>100</element>

   <element>200</element>

    <element>300</element>

    <element>400</element>

</in>

 

I needed to create & store context variables like this.. 

 

var://context/TEMP/element1

var://context/TEMP/element2

var://context/TEMP/element3

var://context/TEMP/element4

 

Each context variable contains <element> node. So if incoming number of element nodes are 10 in the request, total 10 context variable needs to be created.

This is the XSL that did work for me.. 

 

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" xmlns:dpconfig="http://www.datapower.com/param/config" xmlns:str="http://exslt.org/strings" xmlns:dpfunc="http://www.datapower.com/extensions/functions" xmlns:regexp="http://exslt.org/regular-expressions" extension-element-prefixes="dp" exclude-result-prefixes="dp dpconfig str dpfunc regexp">

<!--

Requirement is to Create & Set Context Variables Dynamically

 

var://context/TEMP/element1

var://context/TEMP/element2

var://context/TEMP/element3

...

..

var://context/TEMP/elementN

('N' decided on the basis of number of incomig elements received (input/element))

-->

<xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>

  <xsl:template match="/">

     <xsl:for-each select="input/element">

     <!-- Capture position of each "<element>" tag --> 

       <xsl:variable name="i" select="position()" />

       <dp:set-variable name="concat('var://context/TEMP/element',$i)"  value="." /> 

    </xsl:for-each>

  </xsl:template>

</xsl:stylesheet>

 

DataPower Logging - LogTargets & LogCategory (Part 1)

DataPower has provided easy way to allow developers to create log entries or capture information related to a transaction. If you are familiar with Java, it's more like log4j where you have appenders and log levels. Before I dive into implementation details, let me start with few important words and their definitions

1. Log Category - DataPower comes with few default log category and also allows to create new ones. Think log category as tags. Say if you want to generate some logs, you can tag each log entry. The purpose is to differentiate logs by applying tags. Say from your XSL you want to generate 5 different kind of logs, then you can just have 5 different tags. Each log entry will be fired with the specified log category.

2. Event priority - is like log levels. DataPower doesn't provide any option to create new priorities. Few examples that comes with DataPower emergency, error, debug, information etc. 

3. Log Target -  Log target capture logs fired by services or object. Log targets are like appenders. You can have as many targets as you want. Targets allow logs to captured in DataPower files system or send to a remote server using protocol HTTP, Syslog, SMTP or NFS. Each log targets subscribes to combination of log category and event priority.

Note: DataPower comes up with a default LogTarget that capture all log categories including customs if created on debug level and mgmt on notice level.

To create a new Log Category, search Log Category from search text box from left menu bar.
Create a new one and apply. In this case, I have create "ProjectLog01".

Now you can create a new LogTarget say "ProjectTarget01". In this log target, I've subscribed new log category with event priority "debug" which means debug or all other log entries above debug will be captured.

For example below xsl:message statement will generate a log for default log category "xsltmsg"


And this one will generate a log for "ProjectLog01" category. LogTarget "ProjectTarget01" created above will capture this entry.



Please put your comments if you have any specific question about DataPower logging. I will cover Probe and some other feature of DataPower in next few postings.

Invalid JSON Number after store://jsonxtojson.xsl transformation

"json:number" is coming invalid after store://jsonxtojson.xsl transformation". We have been facing this issue with JSONX 2 JSON transformation.

e.g. <json:number name="balance"></json:number> "Converts to" {"balance":} which is not a valid JSON.

DataPower implements specs, according the JSON spec an empty string is NOT a number:

http://tools.ietf.org/html/rfc4627#page-4
(because of the "int" it has to contain at least one digit)

You have to make sure that the JSONX you generate follows the spec, then conversion is done without problems.


Alternatively (but that is not a best practice), you could copy json2jsonx.xsl to "local:" folder and modify to your needs.

What is WebSphere DataPower?

IBM WebSphere DataPower SOA Appliances is a family of purpose-built, easy-to-deploy network devices that simplify, help secure, and accelerate XML and Web Services deployments while extending SOA infrastructure. DataPower Appliances contain many specialized hardware components, including ASIC-based IPS, custom encrypted RAID drives, and (optional) hardware security modules. DataPower Appliances operate a single digitally signed firmware containing an operating system and application stack. DataPower's firmware runs on a flash storage device. IBM refreshes and enhances the DataPower firmware image every 10–20 weeks. DataPower firmware is a well performing and highly optimized platform to perform electronic messaging functions. As a result, users cannot run 3rd party applications on DataPower as they would a traditional server and operating system. Instead of a traditional file system, DataPower runs with a collection of isolated virtual file systems called 'Application Domains'. As a result, DataPower can appear to its client connections be any type of network file system with any type of folders and links.

Based on Hardware Model 7198
• WebSphere DataPower Service Gateway XG45
 
Based on Hardware Model 7199
• WebSphere DataPower Caching Appliance XC10 V2
• WebSphere DataPower Integration Appliance XI52
• WebSphere DataPower B2B Appliance XB62
• WebSphere DataPower Edge Appliance XE82
 
IBM ESB portfolio offers 3 different products to its client, one of them in
WebSphere DataPower XI series.
 
1. WebSphere Message Broker
2. WebSphere ESB
3. WebSphere DataPower Appliances (XI52)
Here's a link to the IBM ESB portfolio brochure, which lists additional selection criteria
for all 3 products:
 
ftp://ftp.software.ibm.com/software/websphere/2008_ESB_Portfolio_Trifold.pdf