BPEL (Business Process Execution Language) is too machine-oriented, catering to applications talking to other applications, they say. Most business processes need the human touch somewhere along the line.

Do people need BPEL4People?

Consider these un-automatable scenarios: A process may need an executive’s approval to proceed any further. Work may flow like a river, but it also encounters plenty of waterfalls, dams and locks on the way — points at which humans may need to jump in to keep things moving. Workflows are as unique as the companies that create them, and all have their own points where humans intercede.

That’s why OASIS announced it is forming a technical committee to explore how the proposed BPEL4People standard (WS-BPEL Extension for People) could rectify this. This is a step toward becoming an OASIS standard, and work will commence on both both BPEL4People and WS-Human Task. WS-HumanTask was created by Adobe, Active Endpoints, BEA, IBM, Oracle and SAP.

As OASIS puts it, the Technical Committee “would define: (1) extensions to the OASIS WS-BPEL 2.0 Standard to enable human interactions, and (2) a model of human interactions that are service-enabled.” The case for BPEL4People vision was first laid out in a white paper jointly published by IBM and SAP in July 2005.

BPEL4People and its cousin WS-Human Task is already appearing on the market. Just this past week, Active Endpoints announced that an update to its open-source ActiveBPEL Community Edition 5.0 Server includes implementations of both standards.

For those who want more details on how BPEL4People works, Andrew Doble provides a deep dive into how BPEL4People fits into the architecture.

But, ultimately, can BPEL4People finally bring SOA closer to the business processes its supposed to support? Just as BPEL has taken its knocks over the years, there are conflicting viewpoints on whether BPEL4People can effectively do the job.

Fred Cummins, for one, doesn’t think it can. “BPEL isn’t designed for business users,” he says in a new post, “it’s designed for programmers. …BPEL4People won’t change that.” BPEL does not have a standard graphical representation, and business users have trouble recognizing processes transformed for BPEL.

Instead, Fred calls for a “true business process language must be designed to represent business processes that make sense for the business and can be, in some cases, implemented manually, as well with a BPMS (Business Process Management System).”

Fred favors BPMN (Business Process Modeling Notation), which, he says, “was specifically designed for graphical representation of business processes for businesspeople, and it has been widely adopted by the industry.” Plus, he adds, BPMN is now supported by BPDM (Business Process Definition Metamodel), which also is SOA-friendly.

The BPEL4People-BPMN discussion needs to continue. The bottom line is that we need ways to help bring BPM and SOA closer together, which is the next challenge facing implementations in both areas. The important thing is that vendors and industry experts are obviously recognizing that this is an important piece of the puzzle that needs to be addressed.

I once heard a rumor that there actually is a company with two integration teams that actually meet and talk once or twice a year. Just a rumor, mind you.

Lorraine Lawson brought up the whole issue of silos and lack of communication in a recent post, and the implications for SOA. Namely, that SOA not only requires developers to know what other developers are doing, but that the business know what developers are doing, and visa-versa. Lorraine cites the example of one IT professional who had no idea what his coworkers did all day.

This does not bode well for SOA, which is supposed to automagically bring IT and business into alignment, kept in synch by shared services that break through those silos. Repositories and registries may help open the lines of communication and awareness, Lorraine observes, “but even with these technology solutions, how the IT organization interacts internally and with the business is a common problem for SOA implementations.”

Lorraine points to a recent post by Eric Roch, who has worked with many companies on their SOA implementations, who advocates the establishment of an SOA competency center, supported by an SOA steering committee, that can serve as a clearinghouse and information center for all matters SOA.

The biggest challenge that such an arrangement may help address is the anti-enterprise tendency to build services that only meet the needs of the one silo. “When an IT functional group creates a service, it generally applies to their silo-ed view of systems. Or, given this limited view, they don’t create functionality as a service, but bury the functionality in the siloed application with no interface at all.”

I’ve also heard it said that competency centers also lift SOA matters above the grind of organizational politics. SOA projects can be prioritized according to the needs of the business at large, and not to serve the agenda of one business unit. Essentially, they can be silo-agnostic. (How’s that for a new term?)

However, the challenge is that competency centers are the luxury of the largest companies — small to medium-size businesses usually can’t commit resources or staff time for such an establishment. Of course, SMBs have fewer silos to poke through. But what is needed there is an SOA evangelist who can take some time out of his or her regular job to sell SOA to the business, while keeping the lines open to IT.

Intrepid analyst-on-the-go Tony Baer reports he’s hopping between two simultaneous conferences in Orlando (JBoss and IDS Scheer). The conferences may be sharing the same conference venue, but the cultural chasm couldn’t be wider, Tony observes: “On one side, a bunch of open source developers [the JBossers], on the other, a sober group of business and enterprise architects [IDS Scheer] who deal with what the JBoss crowd might otherwise consider the enemy: IBM, SAP, and Oracle.”

JBoss’ journey from command line to corporate

However, a similar chasm seems to be growing within the JBoss world itself: between those who want to advance the middleware server as a slicker, more GUI-ized corporate platform, and those who love it for its technical compactness (with command-line interfaces, for example).

Tony says that it’s possible — especially since Red Hat acquired the company — that “JBoss is finally growing out of its outlaw heritage.” However, JBoss now has two masters: “the installed base that likes the freedom of being able to monkey around with the appserver without losing support, and the structures of Red Hat Enterprise Linux, where changes to the supported configuration might jeopardize support.”

Tony added that he found “a surprising number of JBoss Server customers who considered Red Hat Enterprise Linux was too bloated, opting for other distributions instead. Aside from Hibernate, JBoss still has its work cut out getting adoption of the portal, rules, orchestration, ESB and other parts of the platform.”

This will be the challenge going forward as Red Hat JBoss seeks to offer an alternative to the Oracle/BEA, IBM, and Microsoft SOA stacks. But this is truly JBoss’s time to shine as well, in the role of market disruptor. The small to middle market — long underserved or unserved by the large infrastructure vendors — is taking an interest in SOA. Larger vendors are only too happy to kick such low-margin commodity business to smaller vendors, but ultimately, this is where all the future lies.

Microsoft which also plays in the low-margin space, has learned how to strike a good balance between its techie and corporate constituencies. Perhaps it’s an example for Red Hat JBoss to follow.

Here’s a quote that’s too good to pass up:

“Let’s be honest, the term ‘SOA governance’ sucks. It reeks of someone else telling you what to do, hectoring you over every little detail of a project. It sounds about as desirable as a colonoscopy with an IMAX camera.” -Michael Meehan

Does ‘SOA governance’ sound too much like ‘SOA politburo’?

I wish I could have come up with an analogy like this. In a new post, Mike looked at all the attention being heaped upon the topic of SOA governance and wonders if the term — not to mention the concept it represents — is too overbearing for our business culture?

“It’s a particularly sticky term here in the U.S.A. We don’t like a lot of governance. In fact, we get uppity when we think we’ve been placed under the yoke of too much governance. We’ll dump your tea in the harbor when that happens. In fact, you can be sure many project teams have formed some unprintable thoughts about governance without representation.”

In other words, does SOA governance have too much of a ring of “SOA Politburo”? (How’s that for a better term?) The trouble is, that may be the perception some organizations have.

Mike’s main beef is the term “governance” itself, of course, but I’m wondering if governance is being sold as a panacea for fixing any and all dysfunctional SOA attempts. Granted, ungoverned SOA would not be SOA at all - Just a Bunch of Web Services tangled up in a Spaghetti Oriented Architecture, with no clue as to what’s being used and what value it’s delivering. But with too much governance, as we’ve seen in the past, end users end up doing end-run around the rules with either sneaky approaches or all-new technologies altogether.

I’ve even heard of cases where SOA governance itself has tended to go too far, strangling the innovation that service orientation and loose coupling is supposed to promote. One vendor executive I recently spoke with said he saw customers pull back on governance when they realized that the restrictiveness stifled the ability to effectively deploy and reuse services. Besides, sometimes rogue services can desirable, and even profitable, too.

I have always thought that simply calling it “SOA management” fills the bill, anyway.

Acquisitions are a dime a dozen (okay, make that a billion a dozen) in the SOA space, and we’ve seen plenty of specialized SOA vendors get swept up by larger infrastructure players over the past couple of years.

From enterprise service bus to integration on demand — the shape of things to come?

But the latest acquisition has a funky aspect to it — the acquirer is a Web 2.0 SaaSy company, Workday, which is acquiring Cape Clear, an ESB company. (ZDNet colleagues Dan Farber, Dana Gardner, Phil Wainewright have weighed in on the news as well.)
Workday offers ERP applications via the SaaS model, and has been quite an interesting story over the past year or so. They call themselves the “on-demand alternative to ERP.” (SAP seems to have taken note, as they announced they plan to offer SaaS-based ERP via “Business By Design”)

How does Cape Clear fit into this picture? According to CEO Annrai O’Toole, this reflects something he’s been talking about for some time — a movement away form Big IT and Big SOA to the delivery of software-based services on an as-needed basis:

“For those of you who may be a little surprised at a hosted applications company buying an SOA and Enterprise Service Bus company, you shouldn’t be. This is really the logical outcome of many of the things I’ve written about …over the last several years… 1. SOA needs to be kept simple and focus on the business side of the house, 2. SOA is about enabling applications, not technology, 3. The future of SOA is tied up with the whole phenomena of ‘On Demand.’”

Workday is leveraging its Cape Clear purchase as “Integration on Demand,” (thank goodness they’re not calling it Integration as a Service, or IaaS), which addresses one of the thorniest obstacles to SaaS — the ability to tie on-demand applications and data with existing on-premise systems. With Cape Clear ESB, Workday says it “can offer both packaged and custom integrations that can be designed and deployed much more quickly than on-premise approaches.”

Not everyone looks upon the acquisition so positively, though. Ronan Bradley makes the observation that no SOA middleware company has ever really seemed to have broken into the big leagues, opting instead to be acquired and re-directed into other pursuits.

Ronan, who was CEO of SOA middleware provider PolarLake, observes that his former company “has repositioned itself to take advantage of a specific niche (Reference Data Distribution in financial services).” Now, he notes, “Cape Clear has disappeared from the ’traditional’ enterprise middleware market to become part of a SaaS play.”

Ronan also notes that the trend has been for smaller middleware vendors to get “squeezed between the industry giants with comprehensive software stacks and service arms on the one hand and [open source software] projects on the other.”

Is this a good, bad, or indifferent thing? Ronan is concerned that the disappearing base of smaller, specialized SOA vendors does not bode well for innovation in this space.

Ironically, if you’re an entrepreneur looking to make your mark with a start-up these days, you’re more likely to launch a Web 2.0-style company (ranging from social networking to mashups to SaaS) than an SOA company. Hence the irony of a Web 2.0 company now buying one of those SOA companies that burst on the scene not too long ago.

One more thought: hopefully, Annrai will continue speaking his mind about innovation and the industry — we need more of that.

Geek & Poke’s Oliver Widder picked up on my post from a couple of weeks back, “Taking the ‘complex’ out of complex event processing,” and provides this scenario on how events trigger actions.

Can a vendor’s offering be too well integrated for its own good? Definitely, when it comes to SOA.

Packaged SOA defeats the very purpose of SOA

Service-oriented architecture is not meant to be “bought” the same way a packaged ERP or CRM system is bought. First, SOA is a philosophy (high level) and methodology (nuts-and-bolts level), and thus can’t be captured in a single product. Second, the idea of buying a “packaged” SOA suite runs contrary to the whole notion of SOA, which is supposed to be about interchangeable solutions from any vendor or service creator. Third, buying expensive packaged apps isn’t the way to “start small” with SOA, which is the best route for many companies.

Nonetheless, plenty of vendors are “service-enabling” their offerings, and thus offer compelling shortcuts to achieving at least some aspects of SOA.

These packages or suites must be tempting to enterprises, since they offer the chance to greatly accelerate SOA efforts in critical pieces of the infrastructure and application stack.

Judith Hurwitz raises some of the pros and cons of SOA buy versus build in a recent post, and talked about the understandable confusion a CIO was feeling toward moving to SOA via packaged application offerings. Among the questions that need to be considered are the following:

“What is the benefit and danger of implementing a package software offering that has all the industry best practices, business process, and middleware integrated together. What are the opportunities and risks of this approach? Likewise, what are the risks of buying piece parts and integrating them together?”

Judith acknowledges that it must be awfully tempting for CIOs to take “the path of least resistance” by buying into well-integrated software suites from the likes of Oracle, IBM, and SAP. However, this is a temptation that must be resisted, she urges. As also noted at the beginning of this post, Judith says SOA is all about creating flexible, modular environments “where it is easier to add or subtract components based on either a new business initiative or a new innovative technology.”

Judith urges IT managers and CIOs to look at packaged software as components in an overall SOA strategy, “rather than the lynchpin of that strategy.” Begin with the overall business strategy and an Enterprise Architecture and work from there, versus attempting to extend SOA out of a packaged application environment.

Judith adds that off-the-shelf SOA solutions would make life much easier, but “unless you are buying a commodity, I think the world is still too complicated for packaged SOA.”

Indeed, a single-vendor SOA environment is an oxymoron, like jumbo shrimp or clicking “start” to shut down a computer. But many enterprises, if not most, have a comfort zone in dealing with a single master vendor, and many tend to stay the course with their vendors’ SOA roadmaps. Will they be willing to venture outside of these comfort zones? Ultimately, SOA is a transformative process reshaping the entire enterprise, and executives are not likely to want to go it alone without a vendor partner taking the lead. Nevertheless, SOA is an important step on the road to independence.

Is it time to fold SOA into Enterprise Architecture (EA)? Is there even really any fundamental difference between SOA, EA, and for that matter, BPM (business process management)?

That’s the question Kyle Gabhart asks in his latest post. (Dave Linthicum also predicted last year that SOA would eventually fold into EA).

At the end of the day, SOA, EA, BPM (and let me add Enterprise 2.0 and enterprise data management) call for similar methodologies, and address the same problems. Are they essentially the same thing?

Kyle notes that at a governance panel at last week’s Open Group conference, members of the audience asked panelists to delineate between governing SOA and governing EA. This proved to be somewhat difficult, as “the line between these disciplines is quickly blurring within many enterprises,” he said.

“We need to look past the labels that are applied and take a more business-focused and goal-oriented approach to education, mentoring, and ultimately solution development. We need to probe more intently with our clients to discover their strategic direction, business drivers, and objectives for one-year, two-year, and five-year timeframes. We may have the perfect service offering for them, but because it is labeled as SOA, BPM, or EA, it may not jive.”

Agreed. Business leaders don’t care if a project is called “SOA,” or “EA,” or “DOA.” They want an initiative that achieves faster time to market, streamlines an outmoded process, or improves front office productivity. Period.

Kyle backs up his assertions with some examples. For example, at one workshop he conducted for the Department of Defense, participants observed that the discussion on SOA principals, methodologies, and best practices “all flowed very nicely” into a broader scope of EA. “The ‘SOA’ label that we had begun with was immaterial to the objective — better governance of enterprise assets and business processes.”

He also observes that his company’s clients often associate SOA very closely with BPM, and visa versa. “It was the same standards, same tools, and the training was about 90% the same,” he said. “The difference was found in the labels attached and the emphasis upon certain key concepts.”

At this point, BPM and EA seem to be in separate camps within organizations, and, significantly, the BPM and EA folks, by heritage, may have tighter relationships with the business, versus the IT-centric heritage of SOA. This may be the main divide at this time. But as Kyle points out, it makes a lot of sense to bring these all together in pursuit of better business performance.

ZapThink’s David Linthicum, who nailed the SOA-business alignment problem so well at last week’s Open Group’s Enterprise Architecture forum, points to another matter creating headaches for organizations: consultants offering expensive “SOA” solutions that end up being far less than promised.

Consultants have a long proud history of overpromising and underdelivering — why should things be different with SOA?

Of course, the past four decades of IT history (and management in general) are littered with horror stories of overpriced consultants overpromising and underdelivering. Expensive consultants are brought in to try to unravel the screw-ups left by previous teams of expensive consultants, and the beat goes on. Why should things be any different for SOA? Or, perhaps, is SOA adding a new dimension to consulting engagements that consultants can’t quite get their arms around?

As SOA has gained steam over the past few years, I’ve often wondered how systems integrators would be addressing this change in their billable-hour business model. The million-dollar integration projects that took months or years to complete could, in many cases, be reduced to days and weeks with standardized Web services interfaces.

Dave says not to get too worried about consultants’ revenue streams, as there are plenty that are now profitably riding the SOA bandwagon. However, organizations risk spending a lot of money and getting little or no SOA in return.

There are four issues Dave is seeing with typical SOA consulting engagements:

1) Consultants — and the people who hire them — can’t or won’t distinguish between SOA and JBOWS (Just a Bunch of Web Services) architectures. (I couldn’t agree more, by the way.) “Be wary if consulting organizations point out their experience in the world of SOA by putting up past projects as proof of their experience,” Dave writes. “Most, if not all, of these past projects are really JBOWS (just a bunch of Web Services) and have no underlying mechanisms to provide agility, which is a core benefit of SOA. …It’s an indication that the consultants don’t understand the core value of SOA, and thus could send you off in all sorts of dangerous and costly directions. So, make sure to hire consultants who understand that SOA is really about architecture, agility, and changeability, and not just about service enablement.”

2) Many consultants are a bit too chummy with vendors. SOA is about achieving the flexibility of being able to remove or plug in solutions from any vendor or source. However, some consultants may take enterprises down the lock-in path. Be wary of consultants that “implement the same vendors and technology each and every time,” Dave says. “…it’s still a de facto practice to take an ‘ESB-oriented’ approach to SOA, no matter what the issues are at hand, or, perhaps an ‘app server-oriented’ approach, or a ‘governance-oriented’ approach.”

3) Many consultants attempt shortcuts around what should be a predefined SOA process. “SOA implementations are complex distributed systems, and thus complex to plan, design, build, and test. The time spent in planning will later pay huge dividends.” However, Dave warns, “many SOA consultants try to use older software development lifecycle (SDLC) and enterprise architecture processes,” instead of a true SOA approach “that requires a specific approach that addresses the unique nature of its architectural patterns… Many consultants attempt to oversimplify the process, rapidly moving through or even foregoing the planning steps. Their main focus is the selection of the technology, or, in some cases, they attempt to force fit a problem with a predetermined technology solution. This approach never has a positive outcome.”

4) Many consultants don’t thoroughly bake SOA into their clients’ business. As Dave puts it, many SOA consultants don’t deliver because “there is the lack of understanding about ongoing SOA operations, and links with traditional enterprise architecture.” SOA is an ongoing transformational process, not a one-time project. “If your consultant has done his or her job, you should have an architecture where the volatility has been abstracted into a configurable domain. As a result, it’s a matter of changing things at the configuration layer to adjust to the changing nature of the business. Therein lies the value of SOA.”

In previous posts, I have speculated as to whether the cross-enterprise transformation required to really make SOA work is too far beyond the scope of the IT departments that maintain the relationships with vendors promoting SOA.

Vendors, IT consultants and systems integrators often have working relationships with CIOs and managers on down the IT silo, but not necessarily with other parts of the business. Yet, SOA calls for the deep participation of other business units. Could many of the failings Dave describes with consulting engagements result from engagements that are too IT-centric? Should these consultants be reaching out more to marketing, finance, and operations decision makers?

Which one of the following will kill SOA projects faster this year?

Lack of money, or

Lack of skills?

Two separate articles out this week talk about double-trouble for SOA in 2008.

Anne Thomas Manes, analyst with Burton Group, is warning that a potential downturn in the economy could pull the rug out from under many SOA projects. According to a report in SearchSOA, Anne warned that “You’re going to see budgets shrink. You’re going to be asked to do more with less. One of the most dangerous things is we’re going to see funding for major initiatives evaporate.” In addition to corporate budget cuts, there will be other pressures on IT, such as as globalization, outsourcing and auditing required by new federal government regulations. (Which will usurp talent away from SOA efforts.)

Economists are divided as to whether an actual recession is coming, or if we’re in for a period of slower growth. But, as I’ve mentioned in previous posts, IT budgets have been mean and lean for years, so IT managers already know how to get around any further tightening, if the economy were to soften. Efficiencies are already being built into most IT operations, via automation, virtualization, open source, SaaS, and, last but certainly not least, reuse and streamlining through service orientation.

IBM’s Sandy Carter sees a different kind of threat on the horizon, however — there aren’t enough people with the right skills to see SOA projects through. IBM recently released the results of its survey of Fortune 1000 executives conducted at last year’s Impact event, and found skills to be wanting. “We’ve definitely found from our customers there is a shortage of SOA skills across the board, not just in IT,” she is quoted as saying. “The SOA skills shortage is really across the entire SOA lifecycle, and includes architects, business, risk management and other professionals that can help companies apply SOA to transforming their business.”

If there is a downturn in IT budgets, at least these SOA professionals will have high value to the business, and thus some job security.

So when you put these two worries — budget cuts and skills shortages — together, you get some very busy and overstressed IT shops. We may find that IT and SOA professionals will continue to be asked to do more and more with less and less, without additional staff resources. (To greatly paraphrase Winston Churchill, never before have so few been asked to do so much.)

As a result, we will see an ongoing juggling act (with more knives and torches thrown in), and some projects will get dropped to the floor. Some SOA projects — even those that don’t cost too much — may simply be relegated to the bottom of the priority list because of lack of time, and lack of perceived urgency. But this inability to adequately staff and fund SOA projects is something we’d see even in a booming economy.

Whether you’re facing the budget axe, or you can’t find people who know SOA, the solution is the same. If your budget is under threat of being pared, Anne recommends downshifting SOA to small, incremental steps — “think big, take small steps” is Burton’s mantra.

IBM’s approach to managing SOA when there aren’t enough skilled people around is almost identical: “You don’t start to try with a huge project. You start with a small project,” Sandy says. “But we also tell them, ‘Start with a big vision. Set a vision that is larger and broader than the scope of the project you chose.’”

Think globally, act locally in SOA — great advice no matter what the state of the overall economy.

IBM just launched a product, IBM InfoSphere Master Data Management (MDM) Server, which will purportedly interface with service oriented architecture. According to IBM, InfoSphere “offers deployment options that give customers the flexibility to use it for both entry-level MDM projects and high-volume SOA MDM systems with one platform.”

Can SOA help make sense of the data that is wildly proliferating across our enterprises?

There’s more buzz around the role SOA can play in getting a better grip on the data that is wildly proliferating across our enterprises. Pfizer has been doing a lot of work in the MDM-SOA area, as reported here a couple of months ago.

IBM explained in its announcement how MDM brings it all together:

“A retail bank can manage a customer’s savings, checking and credit card as a single managed account — a task that banks continue to struggle with from a technical perspective. A bank can also use the MDM Server to tailor product bundles, provide consolidated billing, payments, and statements, and customize service for an account. Additionally companies will gain more flexibility in creating and adding new product data and managing product hierarchies in their daily operations.”

In addition, IBM said its MDM tool supports event notification and compliance management.

So where’s the direct connection to SOA? SearchSOA’s Rich Seeley explored this potential connection with David Corrigan, product manager, master data management at IBM:

“A common problem for SOA implementations where architects and developers find that they need a lot of different data sets from a lot of different sources. A lot of our clients in services-based industries have tried to SOA-enable and have a workflow or a middleware process or business process for opening a new account for example. Very quickly they realize that one critical step is understanding who the customer is, did they buy products from you before, what kind of relationship do you have? There’s no one application that can answer those questions because actually the data is buried in multiple applications.”

If SOA-based services can access such information from one consolidated place, in one common format, then the information is readily available.

Even if there won’t be a direct link between SOA and MDM efforts, the SOA approach to decoupling content and context may provide guidance to the way data should be managed.

Vasco Drecun provides a deep dive into a semantic SOA approach to MDM, and what MDM can learn from SOA. The current challenge with MDM, Vasco observes, is the fact that “our systems are so much proprietary and closed that we do not even dare to question accuracy and adequacy of the meaning of the data within them. We simply close our eyes to the fact that we are really not masters of the master data. Consequently, we cannot really govern it, we cannot control it, we cannot influence its evolution. We can only hope for a comfort of knowing that all master data reside in a single system where we can at least see the field within a table where it is defined, regardless of its true semantic value to support all inputs and outputs of all processes that may need it.”

Perhaps MDM should take a service-oriented approach, Vasco says:

“The only way to really control it is to understand all different meanings in all contexts within which the data adds value to information processing. And at the point of placing the informational reference between the content (data fields in a document type, e.g. ’street number of a ship to address’) and context (input/output in the process flow using that document type, e.g. ‘Transportation Order’ or its clone ‘Expediting Drop Ship Order’) is where we need to capture that link. That is how decoupled content and context work in SOA process modeling.”

Is it time to retrench and get SOA back to basics? Some discussions taking place at this week’s Open Group Enterprise Architecture conference reflect growing concern that SOA efforts have stalled, or that SOA is now in a funk. Is it time for a shot of new energy or funding? Or maybe it’s time to step back and apply a shot of good old-fashioned business sense.

Don’t send good money after bad; pull the plug on SOA efforts not delivering

Too many times, we’re engaging in SOA for SOA’s sake. In a keynote that helped kick of the conference, David Linthicum said there are many situations were SOA may simply not be necessary. A mainframe system that rapidly processes transactions may be well enough left alone. “You’re not going to increase the speed of your systems by putting a layer on top of it,” he said. “SOA success means applying SOA where needed. But if it ain’t broke, don’t fix it.”

Companies should pull the plug on SOA efforts not delivering ROI, David said. He also repeated his prediction from a few months back that that SOA would eventually fold into Enterprise Architecture. It only makes sense, he said — companies are discovering that they can’t have two separate processes — “SOA is EA and EA is SOA.”

Tony Baer’s read on David’s keynote also points to getting back to business fundamentals: “With undercurrents as to whether threats of an oncoming recession are taking its toll on SOA budgets, or whether there is what Gartner terms a ‘trough of disillusionment’ afflicting SOA adoption, Linthicum stated to a room of enterprise architects that you have to start with an ROI case.”

I had the opportunity to join Tony and David in a panel discussion following David’s keynote, in which we performed a “reality check” on the state of SOA. We were joined by moderator Eric Knorr of InfoWorld, along with Tom Morgan (AutoDesk) and Chris Harding (Open Group).

In this panel, we also examined the matter of SOA efforts getting stuck, a theme that flowed from David’s keynote. Our panel explored SOA’s uncertain relationship with other disciplines. It was agreed, for example, that business process management is a key element of SOA and visa-versa. However, we also agreed that there is still a wide chasm between the two. Tom Morgan observed that while his organization, AutoDesk, has had a far-reaching SOA effort underway for a number of years now, they “punted” on applying BPM to the effort.

The panel also discussed the issues that linger around governance, especially in the areas of registry and repository. Tony Baer observed that there were still too many issues at the metadata level, and vendors keep pushing registry/repository as a panacea for governance.

Mashups and Web 2.0 are another area colliding with SOA. Dave Linthicum stated that mashups are, indeed, a viable part of SOA taking place within organizations. However, Eric Knorr said he has heard many organizations are not keen on letting data in or out of the corporate firewall.

SOA can work hand in hand with data management, and Tom pointed out that AutoDesk has been very effectively employing SOA-based services to pull, rationalize, and cleanse customer support data from across the enterprise.

It’s only natural that vendor-sponsored research would show an impending “crisis” in one form or another that, of course, the vendor’s products can purportedly solve.

That’s why I was surprised to see an SOA survey from one vendor that said, basically, everything is honky-dory, even among non-customers in the survey.

In fact, the survey, commissioned by SOA vendor AmberPoint and based on responses from 330 companies, finds more than 98% of the respondents with SOA in production rate their SOA implementations as “successful” to some degree.

Thirty-eight percent said their projects were completely successful, and 60% described their projects as “partially” successful. Most respondents (282) are not AmberPoint customers, and to the vendor’s credit, it is publicizing this high success rate — it usually is in a vendor’s best interest to play up the crisis aspect. Some would have taken that 38% complete success rate number and tacked the word “only” in front of it.

In fact, of those who have deployed SOA applications, only 1.5% report that their resulting systems were “not successful.” None, zero, reported that their SOA effort resulted in a total “fiasco.”

If this survey is even only half right, then this is encouraging news for those that want to drive SOA forward in their organizations.

However, when a survey or poll comes in showing a 98% success rate with something, you have to look twice and three times at the survey to make sure it isn’t too good to be true. Did AmberPoint apply a rigorous, uniform definition of an “SOA deployment,” or did it rely on respondents’ perceptions that they had “SOA” going on?

AmberPoint pointed out that only end-user organizations — not vendors or systems integrators — were measured. Plus, the survey was conducted among people who should be fairly savvy to SOA: “a database of IT professionals who have an understanding for SOA concepts and methodologies…. a large population of architects, operations staff and developers.”

And, AmberPoint also said, this wasn’t “pure” textbook SOA at play here. More than 90% of the implementations had “non-Web services” or even “Non-SOA components” (AmberPoint’s words) to them. Nearly 70% of survey respondents have packaged applications (such as SAP) in their systems. Nearly 60% stated they use messaging other than SOAP (such as MQ or RMI). Nearly half of the respondents tie mainframes into their SOA systems.

A third of the survey group, 33%, had SOA already in production that covered multiple departments or extended to customers/partners. The remainder were either still in pilot stages or had efforts confined to single departments.The greatest challenge to SOA? Lack of SOA expertise, cited by 68% of the respondents. This tells us that even if SOA-related budgets were cut and scaled back, we’re unlikely to see cuts in staffing. Only 21% were concerned about the costs of SOA, anyway.

Another 59% said they have hit organizational resistance to their SOA efforts, which suggests that even with high success rates, educating business management on the benefits is still a must.

Business intelligence is becoming ‘collaborative intelligence,’ thanks to the confluence of SOA and Web 2.0 approaches.

Business intelligence becomes collaborative intelligence

That’s the theme of a Webinar I recently moderated, featuring noted author Don Tapscott, Molson Canada BI manager Katrina Coyle, and SAP’s Lothar Schubert. (Audio replay available here - registration required.)

Don Tapscott, who broke new ground in 1996 with his book, The Digital Economy: The Promise and Peril of Network Intelligence, and recently co-authored Wikinomics: How Mass Collaboration Changes Everything, said that “Web 2.0 and service oriented architecture are really becoming a new mode of production.” He elaborated that these new approaches are “changing the ways that we innovate, the ways that we make decisions, the ways that we collaborate, and the ways that companies engage with the rest of the world.”

This has very profound implications for business intelligence and analytics, Don continued. “Historically, internal data has been accessible in various limited ways through traditional ERP reporting systems and MIS.” Now, data is accessible from not only across the enterprise, but from across the Internet. As a result, Don also sees business intelligence evolving to “collaborative intelligence.”

He sees the marriage of this new accessible data with the firm’s traditional internal data creating “an unprecedented challenge, as well as an opportunity to gain insight into the behavior of the company’s most important stakeholders, and to translate that knowledge into success in the marketplace.”

That’s because end users now share knowledge across “a Web that’s no longer based on HTML, but XML… We now have a platform for computation. In the sense the Internet is becoming a giant global computer that everybody programs every time they go on it.”

Molson’s Katrina Coyle also credits SOA with reshaping her company’s ability to compete in a fast-changing and often fickle market. “One of the terrific things that we’ve had in the last year is service oriented architecture,” she explained. “We can now deliver information to our business in any way they want…. we can drive information through emails, text, BlackBerries, and widgets. If we have issues anywhere in the supply chain, we can get that information out in real time to supply chain managers.”

Complex event processing (CEP) has been getting a lot of attention as of late, and is seen as the next stage of evolution for SOA projects. Now it has the big vendors — including IBM — promoting it.

How big will business event processing get?

However, IBM wants to take the technical allusions out of the term itself, referring to it as “Business Event Processing.” The renaming to BEP makes sense, since I doubt if a concept that starts with the word “complex” is going to win a lot of converts.

As part of an announcement in which Big Blue said it would be buying AptSoft, a CEP (or make that BEP) tools vendor, Sandy Carter, senior vice president of IBM’s WebSphere division, said that BEP should be at the disposal of, well, the business. “Everybody is talking about complex event processing. We are trying to rename that category, because we believe the real value is in Business Event Processing, the focus on the business. We believe we are elevating for customers something that was a deep technology capability, something that only engineers understand.”

ZDNet colleague Dana Gardner also discusses IBM’s strategy to open up CEP/BEP to line of business personnel.

With its purchase of AptSoft and plans to include CEP (er, BEP) functionality into its WebSphere platform, IBM obviously sees a robust market opportunity emerging. Carter said IBM already has invested a billion dollars in the events processing space, not counting its AptSoft purchase.

How big will CEP (oops, BEP) become?

IBM WebSphere CTO Jerry Cuomo recently said that he sees the SOA-EDA-CEP marriage as one of the most pronounced trends to watch in 2008. As Cuomo explained in an interview: “I really believe [event processing is] the next big thing in SOA…. we’re taking it very seriously.”

Others in the industry agree that CEP/BEP is going to be huge. In a new post, Don DeLoach, president and CEO of Aleri, said that “we are on the dawn of this explosion, and I feel it is surely coming.”

What’s driving the market? DeLoach thinks “the CEP market is going to be huge because it is being driven by a few unmistakable elements and irreversible trends. The volume of data that organizations need to deal with is growing and the time frames in which they need to act are shrinking. CEP is specifically designed to absorb large amounts of data in real-time and analyze that data on the fly.” BEP as well.

Much has been made of Sun’s purchase of MySQL last week for $1 billion, mainly from the perspective of its apparent strategy to corner much of the open source market.

However, Forrester’s Jim Kobelius sees another gem in the whole deal — Sun is acquiring a strong product for delivering service-oriented architecture capabilities.

TechTarget’s Rich Seeley spoke to Jim a few days ago, who noted that beyond the obvious data services MySQL can facilitate, Sun could move to provide “a comprehensive open source SOA offering built around its Sun Java Enterprise System including the Sun Java Composite Application Platform Suite (JavaCAPS) platform based on its acquisition two-years ago of SeeBeyond.”

Jim also said that this could pit Sun and Red Hat JBoss in direct competition for the newly emerging sweet spot of the SOA market — the small to medium size business sector, which is more likely to buy into commoditized and open-source solutions, not pricey high-end SOA suites. This is where the real action will be for SOA in 2008 and beyond.

As Jim puts it:

“As regards its drive to become the leading open-source SOA vendor, Sun is going head-to-head with Red Hat, which, of course, has the growing JBoss Enterprise Middleware, SOA platform/middleware suite. In 2007, Red Hat entered the data services market by acquiring closed-source enterprise information integration/data federation vendor MetaMatrix, then open-sourcing MetaMatrix’s offerings, and then adding them to the JBoss Enterprise Middleware suite. In 2008, Sun is likely to follow Red Hat’s lead in this regard.”

TechTarget’s Rich Seeley has just published the opinions of several SOA market analysts (including yours truly) on what the potential impact of an economic downturn would be on SOA efforts.

If IT budgets got tighter, would anybody notice?

The general consensus appears to be that if things were to get a little tougher, this would be a chance for SOA to really prove itself.

However, the first thought that comes to mind is, if IT budgets were tightened, would anybody notice? IT has been under pressure to do a lot more with a lot less for years now. And SOA has been seen since its current inception as one of several strategies to make things happen within tight constraints.

Neil Ward-Dutton of Macehiter Ward-Dutton notes that no one can really cut back on SOA, since it’s not something you buy — “it’s just a set of concepts.” He urges actively working the SOA angle to keep business transformation on track: “Use SOA to minimize the future cost of change in one or two high-profile areas like this and you’ll demonstrate how IT can minimize costs and still support ongoing innovation.”ZapThink’s Dave Linthicum says that SOA, properly implemented, delivers a very high return on investment — something that will help businesses in uncertain times.

Miko Matsumura of Software AG webMethods echos this sentiment, noting that “SOA is needed more in hard times because you want more of your IT budget to go towards high leverage projects as opposed to reinventing, re-securing, reintegrating and rebuilding the wheel for each new project.” He states a trusim that applies to technology even in the fattest or fastest of times: “So much IT budget goes down the drain on lack of systemic global optimization that IT is dying the death of a thousand cuts.”

My ZDNet colleague Dana Gardner points out that benefits of SOA transcends economic fluctuations. Companies that simply cut SOA because of a tighter budget “are probably not yet serious about SOA. They will be more likely to remain in constant firefighting mode, trying to keep their databases up and email from choking.”

My view is that we will see a mixed scenario, with the companies that “get” SOA continuing full throttle with it, while companies that can’t grasp the concept scaling back. My view is that tough times would enable SOA to prove its mettle, offering a means to streamline processes and reduce redundancies within their development and integration operations. When times get rough, companies get most skittish about new hiring. A case can be made for SOA efforts that enable organizations to continue to run or even expand current operations without adding additional staff to build new services or maintain them.

And, let’s face it, when it comes to budget crunches, the pinch was never lifted off IT since the post-dot-bomb and post-Y2K hangovers of 2001. Even if the economy were roaring with 25-percent growth this year, budgets would still be stingy and stingier. Hence, the interest in SOA, Web 2.0, and open source approaches to problems — these are the best-known ways to get more enterprise bang for the buck.

John Michelsen of iTKO asked an interesting question in his most recent post: “Why does SOA seem to be moving forward a little faster in Europe than in North America?”

Is Europe better with architecture?

This first thought that comes to mind is whether there is hard data that shows that Europe, indeed, is ahead in SOA. Just today, IDC posted a press release that talked about Europe’s growing interest in SOA. The leading sector is banking, in which about 25% have SOA in production, along with 48% with SOA underway or being planned.

But good comparative data between the two continents is hard to come by. Without hard data to compare North American and European adoption rates, all we have is anecdotal evidence.

John speculates that part of the reason European adoption seems higher is that it SOA is often driven by developers working in smaller teams. “Service-orientation can be started on a much smaller scale, and tested pragmatically before rollout to the larger organization. So, SOA there doesn’t need to be an enterprise-level initiative in all cases, it can be something the company dabbles in before making a full commitment.”

Still, others say there is a greater emphasis on enterprise architecture within European companies, and that’s the reason SOA seems stronger there.

ZapThink’s Ron Schmelzer observes that there is widespread belief (which he mainly disagrees with) that “enterprise architecture as a practice is more widely respected and practiced in Europe than it is in the US. Some believe that the IT community in the US is perceived as developer-centric, coding cowboys that care not a whit about architecture.”

Still, he observes that “most large European-based IT shops see enterprise architecture as a way to guarantee that projects don’t go off into the weeds with developers doing their thing without a central coordinating philosophy and organization. In essence, they see a distinct difference between architecture and development, and the roles of architects and developers, whereas that precise topic seems to be an item for debate in the US.”

Again, I am not aware of hard evidence or data that Europe is ahead or even with North America in terms of SOA. Perhaps Europe has a keener sense of reuse and sharing baked into its culture, and this spills over into its management approaches. But, make no mistake about it, SOA is clearly a global phenomenon — in fact, Asia is probably seeing the fastest SOA adoption rates of all.

When IBM acquired Gluecode, many industry watchers saw it as the ultimate statement that open source was a force to be reckoned with in the SOA and middleware space. And, it was a smart move by IBM to embrace this new disruptive force that promised to bring SOA to underserved and unserved markets.

Joseph Ottinger just posted a tidbit over at The Server Side that suggests IBM may be letting the Apache Geronimo project — which was productized as Gluecode, which became IBM’s WebSphere Community Edition (CE) — wither on the vine. (Apache Geronimo is an independent community project, but a number of committers are employed by IBM.)

“An anonymous conversation with a Geronimo user who’s been paying close attention to Geronimo’s development yielded an interesting statement: Geronimo’s days are numbered. The reason offered was IBM’s sponsorship, in which a large number of the Geronimo committers were hired by IBM, and for whatever reason, innovation ceased within the project.”

I checked with Brenda Haynes, director of WebSphere open source strategy and development for IBM, about the allegation, to get a sense of what exactly is the depth of IBM’s continuing commitment to Geronimo and CE.

Haynes said IBM is not scaling back on its commitment to WebSphere CE, noting that interest continues to run high among customers. “There’s a lot of data that suggests Geronimo is having fantastic acceptance and interest out there,” she said. “The download rate is healthy and continues to grow. When Geronimo community released its [Java] EE5 support, the downloads really spiked. As the IBM team working on that, we’ve been thrilled to see the acceptance of Geromino, and continued growth of Geronimo.”

Haynes also responded the question raised in Server Side about IBM scaling back on the personnel committed to Geronimo and CE development, stating that Big Blue continues “to have some of our best folks working on Geronimo. We have a commercial product based on that code base, and there’s no change in our strategy for us.”

However, Ottinger speculates that IBM may be concerned about CE — available for free download — usurping sales from its commercial WebSphere Application Server editions.

Haynes brushed off this allegation, stating that IBM commercial app sales are doing fine, thank you. IBM saw more than 25 percent growth in app server sales from its commercial offerings, and if anything, CE helped drive interest in IBM’s more sophisticated app server offerings. “We feel that Community Edition fits a very important part of our family story,” Haynes said. “There are customers that are looking for a certain level of app server. The characteristics might be size, might be startup time, might be bells and whistles. Community Edition fits well for a number of customers that have specific application needs.”

Ottinger states that with at least 19 out of the 43 committers working on Geronimo, giving IBM heavy influence over the project. He notes that if Geronimo’s development “is being driven by IBM’s business interests and not the community upon which it’s supposed to rely, then its days are indeed numbered. All it will become is an ‘also-ran’ for IBM, a product from which IBM could use the best developers and ideas, while gaining the promotional advantage of Apache participation.”

The future action within the SOA space is going to be within the underserved and unserved segments of the business market — a majority of companies at this point. As this kind of disruption evolves, the vendors servicing the high-margin business continually get chased upstream. IBM’s CE approach is an interesting strategy by a high-end vendor to capture at least some of the vast untapped future-SOA market, which will belong to the open source and commodity players.

Alistair Bathgate responded to Dave Linthicum’s post on SOA being on the budget chopping block at some companies with the observation that the managements of many companies — which famously think short-term anyway — may become really obsessive about immediate returns on projects (emphasis mine):

“There is no point in demonstrating that your $25M investment will pay back handsomely over 10 years. You must find ways of proving that your project will deliver incremental short-term benefits so the longer term project funds itself and delivers ongoing business value as well.”

Dave agrees that “even during good times demonstrating short term tactical ROI, along with longer term strategic ROI, is just a good idea to keep the SOA momentum moving along.”

Perhaps, in companies that may get hamstrung by budget issues, the time is ripe for Guerrilla SOA — which focuses on short-term wins on specific projects, versus attempting more grandiose rollouts that get the unwanted attention of corporate bean-counters. Guerrilla SOA is all about well-targeted, lightweight engagements to address specific business problems.

Mashups, SaaS and Web 2.0 can be tools in the Guerilla SOA arsenal. For those looking for tips on how to operate “under the radar” of the suits, Jim (World Wide) Webber shows you how to be stealthy yet productive in this video.