The Role of SOA Architect

I've read several posts about what people feel the role of SOA architect is or should be, and can generally agree with a portion of most of them.  Traditionally, a technical (e.g. software) architect is responsible for taking requirements provided to them and putting together an architecture or design that best fits the project.  In my experience, due to a lack of understanding on what SOA is, the role of SOA architect is generally relegated to technical/software architect.

However, this is not what the role should be in my opinion.  A primary driver for adopting SOA in an organization is to support business agility and re-usability.  Based on these characteristics then, it seems apparent that the SOA architect must be involved in understanding the business, business drivers, and overall roadmap to make the best decisions that support re-usability and business agility.  This requires the SOA architect to be involved in the communication of project, product and program roadmaps.  Sadly, this seems rarely to be the case from many organizations, where requirements get thrown "over the fence" for service developers to create services, without any design discussions of how this service fits into an overall portfolio, reusability of the service in other projects, etc.

I believe it's due to these situations where we get a large amount of services in production environments that are not tracked/monitored or cataloged in a repository, and then hear how SOA doesn't work and is dead.  I'd love to hear others' experiences as a SOA architect or with a perspective on the role of SOA architecture whether good or bad.

Oracle Service Bus IDE (Oracle Enterprise Pack for Eclipse) Installation Demystified

I've seen several sites that explain some steps on installing the Eclipse IDE, but nothing that presents an approach for knowing which version of the OEPE is required for which version of the Oracle Service Bus.  Hopefully you find this helpful and by "learning how to fish" can give a more sustainable approach to getting developers up and running in the OSB IDE sooner than later without any issues.

First, this is assuming you're installing the OEPE for 11g Oracle Service Bus.  With 12c the plan is to have Oracle Service Bus development available in JDeveloper 12c, and hopefully this process is much more straightforward and less confusing.

Find the OSB Installation Guide matching the version of OSB you're developing against.

This sounds pretty obvious, but if you find the correct installation guide for the correct version, there should be a reference to the version of OEPE to download in the "Obtaining the Software" section.  There should also be a handy link to the downloads page.  Follow the general instructions for installing the software provided by the link on that page, with the following key point:

• At a minimum, you'll need to select the OEPE along with the Evaluation database.  The other options are not required for just setting up a development environment and testing on a remote OSB server.  Of course, for local testing you'll want to install Weblogic server in its entirety and probably Coherence as well.

This will install the correct version of Eclipse to support your specific OSB server, along with general server-side libraries required.  This does not include the actual Oracle Service Bus binaries and libraries, so at this point you'll be unable to do OSB development from the Eclipse installed.

Download the OSB Version needed and install

From the Oracle Service Bus downloads page, find the version of OSB needed and download the binaries under the "Product Installation" section.  Note that you just installed the prerequisites, so that step is not needed.  Follow the instructions to extract this .zip somewhere on your machine and start the setup executable.   Note that if you don't have an updated version of Java available on your path, you'll need to start the installer using

./setup -jreLoc {Path to Java home directory from previous step installation}

Hope this helps clarify the general installation process for the OSB IDE (OEPE).  After this step, you should be able to start the Eclipse editor under the directory used in the first step, in the  "oepe_{version}" subdirectory.  Good luck!

Working with XML payloads in JMS queues from Oracle Service Bus

For some this may seem intuitive and obvious, but wasn't for me as an experienced Java developer for many years.  Having worked in JMS I recalled that the message types supported generally included Text, Object, Bytes, Stream, and Map.  So when I wanted to create both a publisher and subscriber in Oracle Service Bus to a queue I naturally configured each to use a "Text" message type.  This caused me issues in that the queue subscribing Proxy service message flow needed to have a Replace action early in the request pipeline that changed the $body from a text representation of the XML in the queue to an actual XML document using

Replace . in body with {fn-bea:inlinedXML($body)}

This does work.  However, I later realized that OSB has an additional layer of interpretation on top of JMS where if you configure the message type in both the Proxy service subscriber as well as the Business service publisher to use an "XML" message type, OSB will naturally handle and interpret the XML payload in the JMS queues appropriately, greatly simplifying your life.

I hope this helps and saves you some time in getting OSB working well with JMS.  Enjoy!

Implementing a RESTful intercepting filter pattern within Oracle Service Bus

With a RESTful design for web services, you model your primary objects using a URI hierarchy, and leverage underlying HTTP (in most cases) methods to perform CRUD operations on these objects such as getting them, deleting them, and creating/updating them.  And while I am not espousing that HTTP methods map cleanly to CRUD operations, they do conceptually make it easier to speak about various CRUD operations on objects.  A common practice is to group objects under a functional umbrella URL hierarchy, which makes the object relationships more obvious and allows versioning across a group of RESTful services that share common functions.  However, in my experience trying to map this hierarchy cleanly in a service bus isn't always the easiest or most maintainable solution.

What does loose coupling really mean?

Specifically with integration, I commonly see references or hear the expression loose coupling.  Often the writer speaks about virtualization or abstracting the implementation of the services with something like an enterprise service bus.  However, that alone is not enough to capture the essence of loose coupling to me.

So what is loose coupling then?  Well, in integration I believe it means that 2 or more communicating systems do not communicate directly with each other (no direct binding to addresses, i.e. Enterprise Service Bus) and they use some intermediary component(s) that functionally represents the intention of the integration in both network protocol and data ontology.  That is, client systems communicate with a middle component or service that represents an enterprise proxy for the implementation of the actual service, and uses a canonical data format specific to that industry and purpose.  

For example, if developing an integration solution for human resource management, one might expect the canonical data format for the enterprise services tier would use an industry standard where possible, like HR-XML, to represent the data objects, attributes and relationships.  The services implementing these enterprise operations may be tied to a specific ERP implementation or written in .NET using a custom, scaled down XML format, but at an enterprise level the services utilize standards and separate the client from the implementation in both protocol, location and data format.

Agile Adoption Words of Wisdom

This is an excellent article about organizational change required to truly adopt an Agile methodology.  It isn't simply a development team methodology or toolset, but a organizational decision to change many aspects of how business gets done.

TechWell | Agile Is Not for Everyone (and That's OK)

Oracle SOA Suite 11g Composite Fails to Load After Restart

We had an issue at one site where a specific AQ-adapter initiated SOA composite was constantly failing to load properly, leaving the composite in an indeterminate state, where it couldn't be undeployed or redeployed.  This was perplexing since there were other composites subscribing to other AQ queues that were not experiencing this startup failure.  At first this was just chalked up to a poorly installed instance of SOA Suite, but after further examination the cause became crystal clear.

When I compared the source code of the failing process to other similar ones that weren't failing on server restarts, I could tell that the issue was related to how this particular composite was calling another SOA composite for error handling (which all of the SOA composites call).  In the failing composite, it was referencing the WSDL using a URL retrieved from the Enterprise Manager console for that composite, e.g. "http://{server}:{port}/soa-infra/services/{partition}/{compositeName}/{serviceName}?WSDL".  By referencing the WSDL this way, it requires the composite be running and loaded prior to this composite requesting it, since it's requesting the WSDL from the composite itself, and not just referring to the WSDL as a static document.  There is no determinate way to predict which order SOA composites will be loaded by the infrastructure, so doing it this was typically caused issues with the composite to where it would need to be recompiled as a different version and deployed again each time the server is restarted.

To solve this, there is a preferred way and an alternate way of the above solution.  The preferred way is to store common artifacts in the Metadata Store and reference them from your composites where possible, so utility composites like an error handler's WSDL would definitely fall in this category.  For this simple project, it was determined to instead reference the WSDL as a static document living in the infrastructure independent of the composite being loaded.  To do this, we changed the WSDL URL being referenced to http://{server}:{port}/soa-infra/services/{partition}/{compositeName}/{WSDLFilename}wsdl in the composite.xml in the import section, as well as the reference/ui:wsdlLocation attribute that references the service.  However, the previous format for the URL was left in the section in the same .  

To be clear, using MDS to store these common artifacts is a better solution, but for a down and dirty quick solution this is an alternate option.

Oracle Fusion Middleware-Inserting large XML into XMLType columns

A couple of years ago I ran into this issue with Oracle SOA Suite 10g, and posted on this blog how to resolve it.  Which is great since I used basically the same solution in 11g and it works there as well, only it doesn't require any server changes at all.

If you're trying to save a large XML payload into an XMLType column type in an Oracle database table from a DbAdapter partner link (JCA Adapter in OSB), and receive the error:

ORA-01461: can bind a LONG value only for insert into a LONG column

 You should only need to edit the mappings.xml for that partner link, navigate to the column that is an XMLType in the database, and change the xsi:type from "direct-mapping" to "direct-xml-type-mapping".

Then recompile your SOA composite (or update the OSB mappings file) and redeploy and test.

It should now support larger-size XML payloads!

The Importance of Guiding Principles

Whether speaking about enterprise architecture or design within a project, guiding principles are paramount to keeping your focus on what's important to guiding decisions.  Some examples of these would be software re-use over creation, and simplicity over flexibility to make things more maintainable.  It's important to establish what your guiding principles are prior to having multiple teams making decisions either based on uniform principles or using their own discretion.

This importance to me was illustrated clearly with a little-known rule in the NFL.  As a big NFL fan, you would expect with their use of instant replay and reviewing all scoring plays and turnovers, their guiding principle would be the correctness of ruling plays that directly affect the outcome of the game.  However, as reinforced by an ill-fated decision made recently by the coach of the Detroit, if you throw a challenge flag on one of these plays, it renders the play un-reviewable and your team gets penalized 15 yards.  I'm guessing the purpose of the rule is to ensure coaches don't try to become too demonstrative against the officials, but the end result is not getting the play called correctly in some cases.

Had they determined a guiding principle was to ensure plays are called accurately (e.g. accuracy over punishment), this rule would never have been ratified, which has now resulted in a situation the possibly changed the course of a game for a team.  Having a handful of these guiding principles, a small enough number that allows remembering them clearly across teams, will ensure your organization is uniformly making decisions that can impact projects going forward.

Using Spring Portlet MVC 3.0 in Oracle Webcenter Portal (Spaces)

You may find yourself in the same quandary as me.  I want to stand up a portal quickly at an Oracle shop(client), so Webcenter Portal is currently the defacto choice as the solution, and in this case Spaces has been chosen for its ease of setup and collaboration (including built-in integration with BPM).  However, the client is not an ADF development shop, but is familiar with another web development framework like Spring MVC. Since the only way to expose portlets in Webcenter Portal 11g is using WSRP (Web Services for Remote Portlets), it shouldn't really matter what framework you use to develop the portlet as long as it is deployed using WSRP you can register it with Webcenter Portal and use it as you desire.  Note that there are plenty of more informed blogs and websites that provide the argument whether to use Webcenter Portal: Framework vs. Webcenter Portal: Spaces, as well as whether to extend your portal using either WSRP Portlets or ADF task flows.  I will defer to those sites on providing you those decisions to make, but if you have selected WSRP Portlets, here's how you can use your favorite IDE to develop them and deploy them properly to Webcenter Portal: Spaces.  Doing the same with Framework would entail similar steps, you would need to create a WSRP connection to the portlet producer from within JDeveloper and add the portlet to your page through JDeveloper instead of the Spaces web application. As a quick overview, I tried determining how to WSRP enable my portlet for some time before I stumbled onto the documentation within Oracle's 11g docs describing how this can be done via script as a pre-deployment step for any JSR-286 compliant portlet.  This instruction is within the administrator's guide and not the developer's guide, which made me realize their intention of separation of roles for deployment.  So I will document how to get a portlet into Webcenter Portal in several steps based on role to be consistent with this intention:

1. Developer creates JSR-168/286 compliant portlet using their IDE and whatever framework they want
2. Developer packages portlet up as EAR file for deployment
3. Deployer updates EAR file using provided script to expose JSR-286 portlet as WSRP Portlet Producer EAR, and adds weblogic.xml to embedded WAR to specify shared library to include (otherwise deployment FAILS)
4. Deployer deploys WSRP .ear to Webcenter Portlet Producers server.
5. Webcenter Portal: Spaces Administrator registers WSRP v2 portlet producer in Webcenter Portlet: Spaces using WSDL URL.
6. Webcenter Portal: Spaces page designer adds portlet to desired page and sets any default configurations.

Oracle Webcenter Portal 11.1.1.6 VirtualBox: Installing Webcenter Portlet Producers

Overview

In order to integrate some existing web site content/functionality into your Webcenter Portal, you'll likely need to have the provided Portlet Producers Omniportlet and Web Clipping portlet installed in your Webcenter virtual machine.  These make integrating with existing sites much easier and quicker than a home-grown solution, but will need to be installed if you're using the Oracle VirtualBox VM that includes Webcenter Portal.

Prerequisites

First, you'll have to download the VirtualBox VM that runs Webcenter Portal from Oracle at http://www.oracle.com/technetwork/middleware/soasuite/learnmore/vmsoa-172279.html.

Second, you should have already installed it, and then started it up per the instructions at the site above.  In my case I have configured the "domain" wc_spaces, since I want to setup to use Webcenter Portal: Spaces locally.

Take a Backup/Snapshot of the VirtualBox Image

Of course nothing will go wrong, right?!?!  But just in case, make sure you at least snapshot your VM prior to following these steps.

Adding Webcenter Portal Portlet Producers

Since the virtual machine does not include the Omniportlet and Web Clipping portlet out of the box, you'll need to install these and integrate the new WC_Portlet server installed into the VMConfig application Oracle ships with the virtual machine.

Odd numeric issue with XSLT subtraction

On my project that automates a business process within Peoplesoft, we recently ran into an issue where we were updating a Component Interface with a number representing U.S. dollars and it complained about the length.

Upon investigation I could see that for some reason we were trying to insert the value 1.48999999997 into a field that only supports 2 digits past the decimal point, or a scale of 2. So this indicated to me 2 issues with our project:

1. The Java web service we wrote to call the Peoplesoft component interface did not handle erroneous data being sent in, and
2. Somewhere in our automation code before this we were creating a bad currency amount.

To solve the Java web service issue, I noticed our interface classes used both floats and doubles for the currency amounts. And since the Peoplesoft component interface classes required BigDecimals for all the amount fields, I figured the BigDecimal conversion was likely the issue. The code was converting these using the BigDecimal.valueOf(long) or BigDecimal.valueOf(double) methods. While stepping through a JUnit test case I wrote, I could see that the amount was not getting scaled or rounded at all, so we were writing 1.48999999997 into the field causing the error. After revisiting the javadocs for BigDecimal to refresh my memory, I could see we likely needed to simply call the setScale(scale, RoundingMethod) to ensure we didn't try to insert such values in the future. After the change I unit tested it again and could see that now we could handle all the various data values sent in and correctly scale and round the value accordingly.

The other issue was tracked down by my fellow developer to an XSLT that was doing a simple subtraction of 2 currency values, but for some reason was creating this strange value. In this test case the original numbers were similar to 2450-2448.51, which was resulting in 1.48999999997! I figured somehow this was related to the BigDecimal class as well, behind the scenes used by the XSL engine in OC4J (Xalan?). As a workaround, we multiplied the value by 100, rounded it using the XSLT round() function, then divided again by 100 to ensure the correct scale was applied. While ugly it at least functioned as expected.

Using DBMS_LOCK package to implement singleton pattern

One thing that is inevitable as you start replacing batch-oriented integration strategies with more real-time, transactional strategies like BPEL and ESB, especially in a clustered environment, is that you'll run into APIs and database procedures that worked fine in a batch mode, but in a multi-threaded real time integration start breaking.

Using BPEL's Custom Indexes for BPEL Console Searching

Recently I had a request to store unique identifiers (nothing in violation of SOX requirements, a custom number only recognized at this client) as one of the custom indices in BPEL so associated instances could be searched using the BPEL Console.

While this isn't a difficult thing, as the API is well-defined and easy to use within an embedded Java activity, I ran into some things that may save you some debugging time in the future, or myself when I search my own blog later to "remember" how I got it working as expected.

This blog posting will be short and sweet. The one caveat that I found was if you have either an XML-based or Message-based variable, be careful with the getVariableData function, as it will generally return an Oracle XMLElement object, so you'll need to get the text from it and not call the toString() function of it when setting the custom index. I've generally found it much easier to create a Simple-type variable of type String, then use a Copy operation with an Assign activity to get the value from within either an XML-based or Message-based variable into the simple String variable, then in the embedded Java it's easy to use getVariableData(variableName).toString() to set the custom index.

Publishing large XML docs from BPEL to an XMLType column

As is often the case, this was purely a trial-and-error experiment that I got working in a specific way, but there may be other ways as well. In other words, use these instructions at your own risk, as I assume no liability for any problems these may cause in your environment.

Communicating with ActiveMQ 5.2 from Oracle AS 10.1.3.4

First, I have to say this was a very frustrating task. This was for a customer project that requested we create a BPEL process in 10.1.3.4 that listens to a queue in Apache ActiveMQ 5.2. Naturally, I began with a Google search for someone else who's done this.

The search revealed 2 interesting posts: nabble.com, which appears to cover exactly what I need to do, and Oracle ActiveMQ JCA adapter with OC4J 10g How-to article, which is more geared for using the ActiveMQ JCA adapter from within a custom J2EE application. My preference is to set it up to use the JMS JCA adapter as a third party queueing provider rather than setting up a new resource adapter entirely, especially since I wasn't sure how to even setup a new resource adapter in OC4J outside of a custom J2EE application per the article above.

Performance Tuning Oracle BPEL

First, there are many sites that list some psuedo random steps that are related to performance tuning, but very few indicate why you should be doing them. While I can't answer why for all of them, I can provide the benefit of experience at least.

Requirements Documentation Blues

I may be one of the few developers who actually somewhat enjoy documentation. After all, I don't know about your memory skills, but I can't remember what I had for lunch a week ago, let alone why I designed things the way I did on a software project 9 months ago. In my estimation, the biggest reason projects fail is the lack of communication, which is fostered by good documentation.

Good documentation to me doesn't need to be verbose, just enough to capture what needs to be done and how it will be done. I think most information on a project should be captured in diagrams, and a quick scan of the diagrams should give a quick and concise view of the architecture and design, as well as the functional requirements.

Providing diagrams early in the requirements stage, at least as use cases, provides a great deal of insight into who are the actors interacting with the system, and in what way. A good use case analysis will also lead to the development of test cases to help validate the system so both developers and business users know what to expect from the system. Human actors can be captured generally using their roles in the system, and external systems can also be represented as actors. Having a comprehensive list of user roles and external systems showing which use cases they affect shows the capabilities and expectations of the system. I've used this type of diagram as simple as having 2 tables following the diagram that describe each actor in one table, then the other one describes each use case in a short sentence, possibly even referring back to some project requirements document for tracability. Or you can break out each use case into a full document on its own, and show pre-conditions, post-conditions, a detailed flow of interactions between actors and the system, and alternate flows for cases other than the "happy path".

Many tools allow creation of a use case diagram, and need not be expensive. You can use free tools like ArgoUML, an IDE supporting UML diagrams like Oracle JDeveloper, or use something more commercial like Visio. The point is that as long as it gets documented and everyone buys off on the scope of work for the project, it makes it much more difficult later on for the business team to claim that your code doesn't support test cases that don't trace back to use cases they signed off on.

Oracle BPEL Database Adapter DMLTypes

First of all, I'm not an Oracle software engineer on the BPEL Process Manager product line, but having worked with it for a couple of years, I will at least provide what I think is going on with the DML Types available for use in the database adapter, since I have yet to find a spot where they've documented them clearly.

What I'm talking about here are the choices available when creating a database adapter partner link in BPEL using a table operation, not using direct SQL. I think the obvious ones that are well understood are Insert Only, Update Only, Delete and Select. The one not as clearly documented is the Insert or Update option.

If you select this option, it creates a WSDL for the partner link that includes a couple of DML (database manipulation language) operations called Write and Merge. The database adapter documentation says it is preferred to use Write over Merge for performance reasons but doesn't elaborate.

On my current project which integrates to a custom table into Peoplesoft, I first tried using the Write operation in my BPEL process. It would consistently create new records in the table since their isn't a primary key or unique index defined using the true primary keys. I then switched it to Merge, and voila it worked. What I suspect is happening is for Write operations, the adapter attempts to insert the record, and if the database errors out due to unique constraint violation it will automatically try the update operation. So if the table doesn't have a unique constraint for your key set, the Write will not work. Conversely, the Merge operation will look at your toplink mappings file and select using the primary key you define in the wizard, and if a row is returned, will perform an update, otherwise perform an insert. The Merge also supports hierarchical database structures, which I wasn't using it for in this case.

Hopefully this helps understand when and why to use the Write and Merge operations of the Database Adapter. Let me know if my assumptions about the inner workings are correct, as this is written from the perspective of an observer and not an inside developer.

Welcome to my new blog site.

I've been using Oracle's SOA Suite for a couple of years now, and finally feel like it's worthwhile to start documenting the tips, tricks and methods for using it in integration projects.

There are several other worthwhile blogs and sources of good information about using the Oracle SOA Suite and specifically their BPEL Process Manager, but I've still managed to run across situations that I couldn't easily find online somewhere else.

Hopefully as I start posting you find these blogs helpful, and if you find another site particularly helpful, let me know and I can link to it from here so the information is easier to find.

Regards,
James Lewis