AtomPub Use Cases in the Aviation Industry

At the XML 2007 Conference in Boston, I introduced my concept of an Integrated Documentation Environment for Aircraft Support (IDEAS) based on AtomPub and OpenSearch. The following are some use cases that illustrate how such an approach could facilitate integration and technical information publishing in the aviation industry:

• Notification. Nancy is an aircraft mechanic. When she gets to work in the morning, she opens her feeds aggregator to get new and updated content from all of the following sources: airframer, engine manufacturer, component manufacturers, FAA, or airline policies. Essentially, Nancy doesn't want to login into the support sites of all those content providers to find out what is new and updated. She wants content pushed to her instead. This use case is implemented with the Atom syndication format.


• Federated search. While Nancy is repairing the hydraulic tank, she wants to perform a single search against all those content repositories. She wants the results aggregated and returned to her as Atom entries, so that she can subscribe to those items that she is interested in and receive updates via web feeds. This use case is implemented with the OpenSearch specification.


• Airline Originated Changes. Judy is an engineer working on a new engineering order (EO) to be performed on the hydraulic tank. The airline's technical documents are hosted by the aircraft manufacturer. Judy uses an XML editor which is also an AtomPub client to post the EO to the remote content repository (an AtomPub server).


• Distributed Aircraft Manufacturing. Future Composites Inc. is a supplier of composite aircraft structures to X-Aero, a major airframer (systems integrator). Future Composites is also responsible for providing technical content in S1000D to X-Aero on those composite structures. After a failed attempt to connect to X-Aero's repository using their SOAP and WS-* interface, Future Composites and X-Aero mutually agree to go back to the basics and use AtomPub and its simple and generic RESTfull HTTP interface to CRUD (create, retrieve, update, and delete) documents to X-Aero's content repository.

The main argument in favor of this approach is simplicity and scalability. I am glad to see that the software industry is moving in that direction. Having been involved in complex WS-*-based integration projects in the airline industry, I believe this new approach is a breath of fresh air. The RESTful approach is also more amenable to agile software development as opposed to the waterfall approach which is typical when the big up front purchase of a proprietary ESB is involved.

Integration projects are becoming critical to the success of new aircraft projects. Speaking about the repeated postponement of the 787 maiden flight in an internal memo send to Boeing employees on April 21, 2008 (and obtained by the Seattle Times) Boeing CEO Jim McNerney wrote:

I expect we’ll modify our approach somewhat on future programs—possibly drawing the lines in different places with regard to what we ask our partners to do, but also sharpening our tools for overseeing overall supply chain activities.

Why AtomPub specifically? Because too many people have been putting the "REST" label on their unRESTful chef-d'oeuvre (HTTP APIs) lately. AtomPub is a good embodiment of the principles of the REST architectural style and a good place to start.

So, what are the key principles of RESTful design:

• Everything is a URI addressable resource
  
• Representations (media types such as XHTML, JSON, and Atom) describe resources and use links to describe the relationships between those resources.
  
• These links drive changes in application state (hence Representational State Transfer or REST).
  
• The only type that is significant for clients is the representation media type, not any other resource type
  
• URI templates (a la OpenSearch) as opposed to fixed or hard coded resource names
  
• Generic HTTP methods (no RPC-style overloaded POST)
  
• Stalessness (the server keeps no state information)
  
• Cacheability
  

Adherence to these principles is what drives massive scalability. Security in a RESTful application can be achieved with any of the following existing solutions:

• XML Signature and Encryption
  
• OpenID
  
• HTTP Authentication
  
• SSL
  

How does the aviation industry gets started with this new approach? This will require leadership from aviation IT specialists, particularly from original equipment manufacturers (OEMs). I don't think that another Air Transport Association (ATA) standard committee is needed. Such standard committees are plagued by vendor politics. By the time they finish their work, someone may have invented a better solution than AtomPub.

In the Java space, the recently approved Java API for RESTful Web Services (JAX-RS) specification greatly simplifies REST development with simple annotated POJOs. Jersey is the open source reference implementation of JAX-RS. Apache Abdera is an AtomPub implementation with Spring Framework integration. The latest release of Abdera features a collection of pre-bundled Atom Publishing Protocol adapters for JDBC, JCR, and filesystems.

The following is an excellent technical article from InfoQ that explains how REST and AtomPub facilitate integration: "How to Get a Cup of Coffee".

Recession-Proof Computing

The past three years have been very exciting at Efasoft. We've delivered value to a number of customers in a variety of industries including automotive, pharmaceutical, homeland security, wireless internet, aerospace, defense, insurance, and customer loyalty management. We've also learned a lot in the process. These projects have allowed us to strengthen our expertise in XML, Java EE, and SOA. In the aerospace vertical where we have a strong expertise, we took the initiative to propose new ideas such as using ISO Schematron to exchange and validate S1000D business rules and the AtomPub protocol and Atom syndication for the efficient exchange of up-to-date aircraft technical data between airlines and aerospace manufacturers.

Going forward, our objective will continue to be the success of our customers. We'll achieve that by researching and implementing best practices as always. We understand that technology must be aligned with strategic business goals, but should also take into consideration the context within which the business operates.

So, key questions that a lot of business leaders are now asking include:

• How can we continue to invest in much needed strategic IT initiatives in the current economic downturn under tight budgets?
  
• Given the high rate of IT project failures, how do we minimize risk?
  
• Which software development methodology can help us deliver quality software on time and under budget?
  
• What are the tools that we can use to help our developers in their work and keep them productive?
  
• Is outsourcing and offshoring the right approach? And if we do outsource, how do we keep control, quality, and our intellectual property?
  

Open source software (OSS) is the answer to some of these questions. OSS lowers the total cost of ownership (TCO). By providing full access to the source code (including unit and functional tests), OSS provides transparency into enterprise software. By supporting standards and open frameworks, OSS allows organizations to avoid vendor lock-in, protect their investments, and find talent in the open job market to maintain and support their software assets in the future (in case the software vendor goes out of business).

SOA and Web 2.0 technologies allow organizations to gain a competitive advantage by supporting business process efficiency and by facilitating collaboration and online communities.

On the SOA front, OSS tools such as Apache CXF (web services framework), Apache Tucsany (SCA implementation), Intallio BPMS (BPMN), Apache Axis2, Mule ESB, Apache ODE (BPEL), and Apache ServiceMix (ESB) have demonstrated their strength in supporting SOA projects in mission critical applications in industries such as banking. Based on carefully researched SOA design principles and patterns, our SOA offering includes the following:

• Business process analysis using BPMN
  
• A model driven development (MDD) approach where appropriate
  
• SOA implementation using emerging standards such as BPEL and SCA (Service Component Architecture)
  
• SOA Governance using open source SOA Repositories.
  

On the Web 2.0 front, we really like the Liferay enterprise portal and the Alfresco document/web content management platforms particularly their built-in social networking features which enable enterprise collaboration and online communities. Document management is one area where we can leverage our expertise in XML and related technologies (XInclude, XSLT, XQuery, XForms, ISO Schematron, and S1000D) to help our customers bring their knowledge assets under control. We'll continue to support the Exist XQuery-enabled native XML database to build dynamic XML content applications. The XRX (XForms, REST, XQuery) architecture with Exist and the Orbeon XForms engine enables what we call "Web 2.0 XML authoring and Publishing". We've acquired a strong expertise in document management for maintenance and operation documentation in the aerospace industry (our traditional forte) and drug related documentation in the pharmaceutical industry.

JBoss Seam is a very compelling application development framework because it not only brings together Java EE frameworks such as Hibernate, JPA, EJB 3, Spring, JSF, Facelets, and Java portlets, but also integrates human workflow capabilities (jBPM), full-text search (Hibernate Search), a business rules engine (Drools), and an integration testing facility. We like the ability to leverage third party AJAX-enabled JSF component libraries such as ICEFaces and Apache MyFaces to quickly create rich internet applications (RIA). However, JBoss Seam is not limited to JSF and can also integrate Flex 3 front-ends.

Going forward, all these open source software will be part of our toolkit as we craft innovative software solutions for our customers.

At Efasoft, we are proponents of agile development methodologies such as Extreme Programming and Scrum. These methodologies are based on practices such as user stories, iteration (sprint) planning, pair programming, unit test first, refactoring, continuous integration, and acceptance tests. Agile programming helps create better software that is also easier to maintain. We've witnessed the success of agile first hand and believe that it can help IT organizations achieve success. For more on Efasoft's approach to quality, see my previous blog Addressing Software Quality Head-On.

Architecting SOA Solutions with a Model Driven Development (MDD) Approach

How do you architect an SOA solution to ensure that it is driven by the business and can respond rapidly and efficiently to ever changing business requirements? A Model Driven Development (MDD) approach can help provide that level of agility. The goal with the MDD approach to SOA is to auto-generate service artifacts such as WSDL, XSD, SCA composites, and BPEL code from the service model.

First, the business articulates their vision for the SOA project in requirements documents or in the form of use cases. Business analysts (BAs) then model business processes that realize the use cases by leveraging the Business Process Modeling Notation (BPMN). With the help of the right tools, the BAs can specify Key Performance Indicators (KPI) such as those required by service level agreements (SLAs). They can also run simulations to validate the proposed business processes.

SOA is indeed all about reengineering and supporting organizational business processes. Back in 1993, Michael Hammer and James Champy made the case and outlined the management framework for reengineering in their book entitled "Reengineering the Corporation: A Manifesto for Business Revolution". Today, SOA is the software architecture that enables and facilitates the reengineering of business processes.

BPMN is an effective tool for BAs (as opposed to UML) because they should only focus on the business and operational aspects of the business process and shouldn’t have to worry about IT concerns such as service loose coupling, reusability, reliability, security, persistence, and transactions. While direct transformation from BPMN to executable BPEL code (so called BPMN-BPEL round tripping) may be effective for simple business processes, it can not always satisfy those IT concerns. More complex business processes will require advanced modeling and coding by SOA architects and developers.

For example, the SOA architect will have to decompose the proposed business process into task, entity, and utility service layers in order to satisfy the SOA principles of loose coupling, reusability, and composability. That will also give the SOA Architect opportunity to apply SOA design principles and patterns and check the enterprise SOA Repository or Registry to reuse existing services or legacy assets.

After decomposing the proposed business process to identify reuse opportunities and address other IT concerns, the SOA architect can then build an assembly of service components based on the Service Component Architecture (SCA). SCA implementation types can be Spring beans, EJBs, C++, Cobol, WS-BPEL, PHP, Spring, XSLT, XQuery, and OSGi bundles. SCA supports different bindings such as SOAP/HTTP Web services, JMS, RSS, and Atom.

The tool of choice for software architects is UML 2.0. In the case of SOA, UML can help abstract the service model from technology-specific implementation details. Basic UML artifacts such as activity and collaboration diagrams can be auto-generated from the BPMN diagrams produced by the BAs to bootstrap the SOA Architect's modeling effort.

To help SOA architects in crafting service-oriented solution logic, a UML Profile for service-oriented design should be adopted. The profile should define a number of stereotypes that can be applied to UML artifacts in order to refine the transformation from UML artifacts to service artifacts.

The automatic generation of the service artifacts from the UML model should be part of the build and continuous integration process which should also include automated tests (for example to ensure that the generated XSD and WSDL are syntactically correct, WSI-BP compliant, and backward compatible).

The benefits of an MDD approach to crafting SOA solutions include: increased development productivity, traceability to business requirements, responsiveness to changing requirements, quality, and overall agility.

SOA in the Java Space: State of the Union

In the Java space, an SOA Architect starting a new SOA project will have to make some strategic as well as tactical decisions regarding which approach and technologies are appropriate for their project. Of course, the SOA project should be aligned with the organization’s long term business goals.

Technically, there is a myriad of specifications to choose from:

• The Business Process Modeling Notation (BPMN)
  
• The Java API for XML Web Services (JAX-WS)
  
• The WS-* specifications including the WS-I Basic Profile, WS-Addressing, WS-Policy, WS-Reliable Messaging, and WS-Security
  
• The Java API for RESTful Web Services (JAX-RS)
  
• The Java Business Integration (JBI)
  
• The Web Services Business Process Execution Language (WS-BPEL)
  
• The Service Component Architecture (SCA).
  

Each of these specifications has its raison d’etre and should be part of the architect’s toolkit. However, I find SCA quite intriguing. SCA defines a language neutral programming model for the assembly and deployment of services. SCA implementation types include Java, C++, Cobol, WS-BPEL, PHP, Spring, XSLT, XQuery, and OSGi bundles. SCA supports different bindings such as SOAP/HTTP Web services, JMS, RSS, and Atom. SCA applications can be hosted in web containers, application servers, and OSGi runtimes. SCA is geared toward the developer and can apply policies such as reliability, security, and transactions to services in a declarative manner. SCA has the support of big players including Oracle, SAP, and IBM. At the time of this writing, Sun Microsystems support for SCA is less than clear (to me anyway).

JBI is implemented by a number of Enterprise Service Bus (ESB) products. JBI defines a runtime architecture that allows plugins such as binding components and service engines to interoperate via a Normalized Message Router (NMR). Binding components (BCs) use communication protocols such as JMS, FTP, XMPP, and HTTP/S to connect external services to the JBI environment. Service engines provide application logic in the JBI environment. Examples of SEs are XSLT/XQuery data transformation engines, rules engines, and WS-BPEL engines. BCs and SEs do not communicate directly. They only communicate through the NMR. IBM, BEA (now part of Oracle) and SAP did not vote in favor of JBI Java Specification Request (JSR 208).

When starting a new SOA project, SOA architects will have to look beyond vendors politics and make some judgment calls about the best approach based on their business goals and functional requirements.

My personal take on this is to adopt an agile approach where new functionalities are implemented in an iterative manner. For example, instead of starting with an ESB infrastructure, a project can start by service enabling existing applications (code-first approach) with JAX-WS annotation capabilities or by creating new services with a contract-first approach where JAX-WS annotated service and server stub are generated from a WSDL. Alternatively, the Java API for RESTfulWeb Services (WS-RS) could be used when a RESTful approach seems more appropriate.

An ESB could later come into the picture in a context where you are connecting to multiple services (often across organizational boundaries) and there is a need for mediation services such as business process orchestration, business rules processing, data model transformation, message routing, and protocol bridging. In that context, JBI provides plug-and-play functionality in ESBs for service engines such as business rules engines and BPEL engines. For that reason, JBI can help avoid ESB vendor lockin (perhaps a reason why proprietary ESB vendors are not backing JBI).

However, SOA architects should carefully consider the benefits of the programming language and binding agnostic service assembly model proposed by SCA. This will be essentially a choice between centralized mediation and decentralized assembly. The Open Service Oriented Architecture (OSOA) group believes that JBI and SCA can actually work together for example to allow SCA components to call JBI components or to use JBI runtime containers to deploy SCA composites.

Decentralized assembly is agile and looks a lot more like the way the web itself works. So I believe that while the JBI model is fine for integrating legacy enterprise applications, new and future service-oriented applications will embrace the SCA approach.

So is the state of the union strong? There is certainly a risk of fragmentation. But choice will always drive innovation forward and that’s what attracts me to the Java platform in the first place.

XML Schema Design Strategies for SOA Projects

The schema modeling and design effort should be an integrated part of an agile approach which implements practices such as user stories, acceptance tests, unit test first, refactoring, short iterations, common code base, and continuous integration.

In the recommended contract-first approach to web services development, the XML Schema and WSDL artifacts are the foundation of an SOA project. For example, with Apache CXF, you can use the WSDL2Java tool to automatically generate a JAX-WS annotated service and server stub from your WSDL.

The first step is to adopt schema naming and design rules (NDR). Industry and government standard bodies like the Universal Business Language (UBL) and the National Information Exchange Model (NIEM) have published such NDRs.

If you’re building a new schema from scratch, then the schema should be designed in an iterative and collaborative manner. During each iteration, add just enough components to your schema to support the specific user stories that are being implemented. As the schema grows, refactor as required.

One option is to start the modeling effort with a domain model in the form of UML class diagrams to facilitate collaboration between non-technical subject matter experts (SMEs), the modeler, and the technical team. An XML schema can then be generated automatically from the UML class diagram with a tool such as Hypermodel. David Carlson, creator of Hypermodel, proposed a UML Profile for XSDs which defined a number of stereotypes that can be added to UML class diagrams to refine the mapping from class diagrams to XML schemas. Alternatively, you could export the UML model to the XML Metadata Interchange (XMI) format and use an XSLT transform to map the XMI into an XML Schema and even an XML instance. This Model Driven Development (MDD) approach to XSD provides agility in the face of constant changes in business requirements.

If you are reusing an industry or government schema such as UBL or NIEM, it is very important to use the right methodology for extending the schema as recommended by the applicable NDR or Information Exchange Package Documentation (IEPD) process in the case of NIEM. Extensions to the standard schema should be clearly defined in a new custom namespace and documented properly. The following are some strategies for extending an XML schema:

• Wildcards xs:any and xs:anyAttribute
  
• Element substitution and abstract elements
  
• Type substitution via xsi:type and abstract types
  
• Concrete Extension (creating a new type by extending an existing type to include additional local elements).
  

The schema should be tested for quality against the NDRs. The US National Institute of Standards and Technology (NIST) has published an XML Schema Quality of Design Tool (or QoD Tool) which combines Schematron and JESS rules (a Java-based open source rule engine) to validate schemas against NDRs.

For unit testing, the XMLUnit framework can be helpful in testing the schema as you refactor and implement new user stories. XMLUnit for Java allows you to make assertions about the validity of an XML document against an XML Schema. The execution of these tests should be part of your build and continuous integration process.

The automatic generation of the XSD code from the UML model should also be part of the build and continuous integration process.

Business rules or modeling requirements that are beyond the capabilities of XML Schema 1.0 should be implemented with an assertion-based language such as ISO Schematron or with the new assertion and conditional type assignment (co-constraints) capabilities in XML Schema 1.1.

Although ISO Schematron and XML Schema 1.1 can be quite powerful when used with XPath 2.0, some complex business rules will be easier to handle with rule engines such as JBoss Drools and JESS. The rule engine can be deployed as a dedicated and reusable utility web service to validate messages.

When modeling XML schemas for an SOA project, careful consideration should be given to the issue of data transformations. When the services don’t share the same data model and XML Schema, there is a need to transform the XML data using technologies such as XQuery and XSLT. This can introduce additional design complexity and runtime performance issues. Data transformations shall be avoided unless absolutely required.

The XML Schema's xsd:appinfo element can be used to capture and keep metadata close to the XSD declarations:

• Metadata such as data transformation specifications
  
• Business rules using inline ISO Schematron rules
  
• Labels, alerts, and appearances of UI components such as XForms controls. This provides the opportunity to auto-generate UI components from your XSD using a transformation language like XSLT or XQuery. Keeping UI and XSD components in sync can be a challenge in SOA projects.
  

Finally, to maximize reuse, an enterprise-wide SOA Repository/Registry should be used to publish, centralize, and discover schema components (see my previous post on SOA Governance tools).

Addressing Software Quality Head-On

Adopting a test-driven development methodology (TDD) and using the right tools can help deliver quality software.

With TDD, you start with user stories and you write acceptance tests for those stories. To implement the functionality for the stories, you write unit tests first, then just enough code to make the unit tests pass, and then you refactor. You repeat the test-code-refactor cycle until the acceptance tests pass.

If your development team is not already using TDD, the first step is to provide adequate training on the concepts and patterns of TDD. One option is to hire an Agile coach or pair junior developers with experienced practitioners of TDD. Next, you need to use the frameworks and tools that will facilitate adoption, keep your developers productive, and provide transparency into the process. The following are tools and frameworks (all are free and open-source) that we find useful:

• Build tool: Maven 2
  
• Continuous integration: Hudson
  
• A tool for configuring, starting, stopping Java containers and deploying applications for continuous integration and functional tests: Cargo
  
• Unit testing frameworks: JUnit 4.4, EasyMock, and JSFUnit (for JSF applications)
  
• Enforcing coding standards: Checkstyle
  
• Detecting bugs, overcomplicated expressions, and suboptimal/dead/duplicate code: PMD and FindBugs
  
• Code review: Jupiter
  
• Test coverage: Cobertura
  
• Web Services functional and load testing: SoapUI
  
• User interface testing: Selenium and Umangite
  
• Integration testing: Fit, DBUnit, DbFit, and ORMUnit
  
• XML unit testing: XMLUnit and Tennison Test
  
• Load and performance testing: JMeter
  
• Profiling and monitoring: JConsole
  
• Analyzing code dependencies: JDepend
  
• Source code documentation: Doxygen
  
• Project tracking and planning: XPlanner
  

By enforcing Java best practices, tools like Checkstyle, PMD, and FindBugs are very helpful for developing high quality code.

Unit and integration tests should be fast, repeatable, and automated. That's why you need to deploy a build tool and a continuous integration server from the start. All the tools listed above should be executed as part of your continuous integration process. You should use a dedicated integration build machine.

DBUnit helps make your system tests repeatable by using XML to insert a specific data set into the database before each test run. Cargo can configure and starts the web container or application server (AS), deploy your application's WAR or EAR file, and then shutdown the container after each test run.

Keep in mind that test coverage reports such as those provided by Cobertura should be used mainly to isolate code that has not been appropriately tested in order to take corrective measures. These reports shall not be used solely to aim at a magical high coverage percentage. Applications that are built on an inversion of control (IoC) container such as Spring are more amenable to unit testing.

For web applications with a user interface (UI) layer such as Struts or JSF, make sure that you exercise the UI functionalities on real web browsers (IE, Firefox, etc.) with a tool like Selenium. Fit and Fitness (the wiki-based version of Fit) are effective tools for integration testing your application's business logic. By integrating Selenium, TestNG, Spring, and Cargo, the Umangite framework makes it easy to write web tests.

If you intend to service-enable the same application, then SoapUI will help with both integration and load testing of the web services over HTTP. The nice thing about SoapUI is that in addition to XML Schema validation, it allows you to specify response assertions by using XPath 2.0, XQuery, and Groovy scripts.

Sun JDK tools such as jConsole, jmap, and jhat can help with profiling and diagnosing memory leak issues. The JMeter Proxy makes load and performance tests easy my allowing you to record a test case.

Are agile practices applicable to XML development (XML Schema modeling, XSLT, XQuery, and XSL FO programming)? You bet. More on that on my previous blog entitled Extreme XML Programming.

S1000D Content Reuse for Aircraft Documentation

One of the justifications for moving to an XML-based S1000D content management system (CMS) is the ability to reduce cost and improve quality by reusing content. In the aerospace industry, hundreds of thousands of pages of maintenance and operation documentation are produced and maintained for every new aircraft project. Warnings and cautions are a good example of reuse in aerospace documentation. They describe hazards that may cause injury or death or damage to the aircraft. For product liability reasons, these warnings and cautions are carefully reviewed and approved by qualified personnel. Technical authors may be required to reuse these warnings and cautions verbatim across all documents. In this blog, I will discuss some principles and practices that facilitate S1000D content reuse.

From a technical perspective, the key to successful reuse in S1000D is the W3C XInclude specification. The S1000D specification does not make reference to XInclude. The reason is that earlier versions of S1000D were based on SGML. Some S1000D CMS still rely on the SGML/XML 1.0 external parsed entity mechanism for implementing reuse. This approach has several limitations and should be avoided. The preferred approach in modern XML content applications is to use XInclude which allows the transclusion of not only whole chunks of XML content, but also elements (addressed using XPath/XPointer) within those chunks. The following are some examples:

<xi:include href="dm.xml"/>
<xi:include href="dm.xml" xpointer="warning-001"/>

In the first example a data module file named dm.xml is included. In the second example, an element with ID value "waning-001" within the data module is included.

Using XInclude in an S1000D content application requires some modifications to the XML Schema used for the authoring of data modules to allow the insertion of xi:include elements. However, these modifications will still produce valid S1000D documents since you're not altering the structure of your documents, but rather simply modularizing the content.

While we are on the subject of inclusion, the XLink specification can be used as a simpler alternative to the XML 1.0 unparsed entity and notation mechanism (another concept inherited from SGML) for including illustrations into S1000D documents.

At the DocTrain 2007 conference in Boston, I gave a presentation on how to integrate training and documentation using S1000D and the Shareable Content Object Reference Model (SCORM) specification. One way to reuse S1000D content in SCORM is to assign a unique ID to all elements in S1000D data modules (DMs) that are reusable such as paragraphs, steps, warning, cautions, notes, tables, etc. This can be done automatically using the XSLT generate-id() function. The instructional designer then searches the S1000D common source database (CSDB) to find and display relevant DMs. She can then use XInclude to include reusable elements from S1000D DMs into SCORM shareabe content objects (SCOs). When this is done, the SCOs are automatically updated when the DMs are updated.

Successful S1000D reuse requires adherence to the principle of context-agnostic content. For example, to make it possible to reuse a warning across multiple documents in different contexts, one should avoid formulations such as "refer to the illustration in the next section" inside the warning.

Enforcing the principle of context-agnostic content can be semi-automated using an assertion-based schema language like ISO Schematron to report the occurrence of keywords such as "previous", "next", "below", etc. The warning shall be routed through a comprehensive review and approval workflow provided by the CMS before final publication. The principle of business rules definitions and enforcement ensures that reusable content is of the highest quality. Consider a dual-purpose data module that is written to be reused by both training and publications. A business rule could require the use of a certain language style (e.g. active as opposed to passive voice) for the dual-purpose data module.

Another principle that can help when the content cannot be context-agnostic, is the parameterization of reusable content. With parameterization, you include variable references in the reusable content that are resolved at run time. The Exist XML database has an elegant way of handling this using a combination of XInclude and XQuery as in the following example:

<xi:include href="warning.xq?var1=material&var2=process"/>

Here warning.xq is a stored XQuery witch is compiled and executed by Exist to return the root element of the warning. The content of the warning depends on the material and process used to carry out the maintenance procedure. var1 and var2 are passed as global external variables to the XQuery.

The issue of content granularity is directly related to the principle of context-agnostic content. Although the data module is the basic unit of information in S1000D, content can be managed at a lower level of granularity. An interested feature of some XML editors is the ability to select an element inside an XML document and convert that element into an XIncluded file. So while a technical author is writing a warning inside a data module, she can pull out that warning as an XIncluded XML file if she determines that the warning could be reusable in other publications.

Another area where XQuery facilitates reuse is the dynamic assembly of content based on product attributes such as applicability, security, and skill level. S1000D has a comprehensive metadata facility called IDSTATUS that can be leveraged to filter content. A good example is applicability filtering. In the case of an aircraft, the applicability of an S1000D maintenance or operation procedure can depend on the following attributes and conditions (among others):

1. Manufacturer serial number
   
2. Aircraft registration number
   
3. Service bulletin incorporation
   
4. Location of maintenance
   
5. Aviation regulations
   
6. Temperature, wind speed, and sandy conditions.
   

XInclude and XQuery can be used together to package content into S1000D publication modules by executing queries that filter content based on metadata in the IDSTATUS.

An important condition for content reuse is the principle of discoverability of reusable content. Obviously, you cannot reuse a piece of content if you don't know that it exists and where to find it. A technical author should be able to query or browse the S1000D CSDB (Common Source Data Base) to find relevant reusable content. To facilitate enterprise-wide content reuse, I highly recommend a CSDB based on a native XQuery-compliant XML database and deployed as a web application. That will allow authors to perform both full-text and structured queries on the CSDB. The query should return a list of data modules or reusable chunks. The author should then be able to select the reusable chunk to automatically insert an XInclude targeting that chunk.

In support of the principle of reusable content discoverability, appropriate metadata should be added to the content. The DMs already have comprehensive metadata in the IDSTATUS section. Reusable content at a lower level of granularity (like a warning) should also have appropriate metadata specified.

An XQuery-enabled native XML database can help with the governance of your reuse initiative by providing powerful reporting capabilities. For example, you can easily run an XQuery to find all documents that contain an XInclude to a particular chunk. This is important for understanding the impact of updates to that chunk. Another potential issue that could require some attention is the versionning of reusable content. Some form of notification mechanism can be helpful to alert consumers to changes to reusable content. This can take the form of an Atom feed to which consumers can subscribe.

It is important to select an XML authoring tool that has good support for XInclude. Fortunately, some commercial XML editors now have decent support for XInclude. However, these XML editors remain complex specialist tools that are often used only by professional technical authors in documentation departments. At one of our aerospace customers, manufacturing assembly and functional test procedures were used to create installation and testing procedures for service publications. To allow their engineers to contribute S1000D content, we designed a light XML authoring application based on an XForms front-end with XML data persisted in a native XML database using a RESTful API.

Any data reuse strategy should look beyond training and publication to identify opportunities to reuse data and streamline processes across the entire aircraft lifecycle.

Spring, SCA, and OSGi

French paleontologist Pierre Teilhard de Chardin once said: “Tout Ce Qui Monte Converge” or “Everything That Rises Must Converge”. I learned this quotation from my father who mentioned to me that it was the topic of his philosophy dissertation at his university entrance exam.

The quotation accurately describes what I see happening in the world of software development with the rise of Spring, OSGi, and SCA.

During the last 30 years, the software industry has evolved from structured design to object-oriented design, POJO programming, and lately service-oriented design. With the rising complexity and costs of software systems, the ultimate goal of this evolution has been the reuse of software assets through loose coupling and service orientation.

Spring is based on the principle of inversion of control (IoC) or dependency injection. With Spring, objects (simple POJOs) are provided with their dependencies as opposed to the objects managing or looking up those dependencies themselves. Spring relies on aspect-oriented programming (AOP) to declaratively manage cross-cutting concerns such as security, transaction, and logging. From a quality and agile development perspective, one big advantage of Spring-based applications is that they are amenable to unit testing using frameworks such as JUnit or EasyMock.

Service-oriented architecture (SOA) exposes application business logic as a set of services that are remotely accessible and reusable across platforms and programming languages. The Service Component Architecture (SCA) has been designed to facilitate service composition. The SCA Assembly Model consists of one or more service components. Service components provide business functions to other components within or outside the module. A composite contains one or more service components and specifies communication bindings and policies such as security and transactions. Like in Spring, the artifacts and the dependencies between them are described using XML.

The Open Services Gateway initiative (OSGi) defines a dynamic service model where components (packaged as bundles) and their dependencies are specified in a service registry. OSGi standardizes the life cycle management of these bundles including deployment, installation/uninstallation, and updates with full versioning capabilities. Bundles can be dynamically started, stopped, or updated without the need for a reboot. OSGi defines a model for publishing, discovering, and binding to services within the same virtual machine (VM).

Spring, SCA, and OSGi are converging to create an environment that facilitates the design and the lifecycle management of software assets that are exposed as reusable services. Software vendors are actively exploring different opportunities to combine these three technologies. The combination of Spring, SCA, and OSGi is already having a transformative impact not only on service-oriented design and application development in general, but also on the application servers and middleware market as well.

SOA and ROA Design Principles and Patterns

I've been compiling a list of design patterns and anti-patterns on Service Oriented Architecture (SOA) and Resource Oriented Architecture (ROA). I find the following resources quite useful.

If you're looking for design patterns in building RESTful applications, the best way to start is to look at the Atom Publishing Protocol (AtomPub) which is a good embodiment of the principles of the REST architectural style. The Google Data API (GData) is a real world implementation of AtomPub. At the XML 2007 Conference, I've proposed a RESTful approach to aviation technical data management called "Integrated Documentation Environment for Aircraft Support (IDEAS)" (more on that on my previous blog RESTful IDEAS).

Another good resource is the book "RESTful Web Services" by Leonard Richardson and Sam Ruby. Chapter 8 entitled "REST and ROA Best Practices" is a must read and also addresses potential REST implementation issues such as asynchronous operations and transactions. Chapter 10 entitled "The Resource-Oriented Architecture Versus Big Web Services" offers ROA alternatives to WS-* specifications.

For SOA design patterns and anti-patterns, here are some useful resources:

• SOA Antipatterns: an article by a group of IBM SOA architects
  
• SOA anti-Patterns: an article by Steve Jones at InfoQ
  
• SOA Patterns: the first draft of Thomas Erl's upcoming book entitled "SOA Design Patterns"
  
• Pete Lacey Criticizes Web Services: an interesting discussion on REST vs. WS-*.
  

I don't believe that ROA is the answer to all SOA project failures out there. However, I do believe that certain requirements and use cases are more amenable to the REST architectural style (more on that in a future post).

SOA Governance Tools

First, an important caveat: SOA governance is not a tool or a product. SOA governance is about people and leadership. No tool will deliver good governance out the box if the human factor is not taken into consideration. However the right tool can facilitate and provide transparency into SOA governance.

It’s important to make a distinction between design-time SOA governance and run-time SOA governance. The main objective of run-time SOA governance is the enforcement of QoS and SLAs. Design-time governance focuses on the enforcement of industry-recognized SOA design principles and patterns.

The goal of these patterns is not to kill the creativity of SOA developers or police their work, but instead to avoid SOA anti-patterns that are known to undermine the success of SOA projects. For example, SOA developers cannot reuse services if they are not aware of the existence of these services in the enterprise. Even if they know these services exist and where to find them, they cannot reuse the services if they don't understand them. Therefore the ability to easily discover well specified service metadata is one good design principle that can help deliver on SOA's promise of service reuse across the enterprise.

One of the key aspects of design-time SOA Governance is the management of the lifecycle of service artifacts and the dependencies between them. This is accomplished through a new breed of tools called SOA Repositories/Registries. The following are what I consider important requirements for an SOA Repository/Registry.

Indexing of XML-formatted artifacts such as WSDL, XML Schemas, Schematron rules, XSLT transforms, Spring and Hibernate configuration files, data mapping specifications, WS-Policy documents, etc. Ideally, users should be able to use languages such as XSLT and XQuery to manipulate artifacts and query the Repository/Registry. Requirements and specification documents should be stored in XML (as opposed to MS Word or Excel) if possible so that they can be processed the same way. The SOA Repository/Registry should sit on a native XQuery compliant database. This would provide powerful visualization and reporting capabilities to the registry. I should be able to run an XQuery search against the SOA Repository/Registry to return all artifacts that contain a reference to a certain XML element, so that I can visualize the impact that a change to that element would have. Automatic detection of certain dependencies (e.g. WSDL and XML Schemas) should be supported as well.

Policy enforcement is also important as artifacts are added to the registry. For example, Schematron can be used to enforce XML Schema best practices. WS-I Basic Profile compliance and XML schema backward compatibility may also need to be enforced.

Support for a RESTful API for all CRUD (create, read, update, delete) operations on artifacts. Ideally, I would prefer support for the AtomPub specification and the Atom syndication format for pushing updates to stakeholders. Competing SOA Registry protocols and APIs include: UDDI v3, the Java API for XML Registry (JAXR), JSR 170/283 (Java Content Repository API), and IBM WebSphere Registry and Repository (WSRR). However, Open source vendors such as MuleSource and WSO2 have adopted AtomPub for its simplicity, while RedHat is building its upcoming SOA Repository/Registry (JBoss DNA) on JSR 170. Mule Galaxy sits on the Apache Jackrabbit JCR reposiroty. The JSR 170 repository model could also be adopted in the Java space as a standardized repository model for SOA registries.

Of course authentication, authorization, audit trails, workflow, and versioning should be expected in any SOA Repository/Registry.

Passportgate: what would you do about it?

The current Passportgate scandal in the US involving the unauthorized access to the passport files of the three presidential candidates got me thinking about information security in enterprise applications particularly records and content management systems.

Ensuring information security requires a multidimensional approach based on technology, process, policy, and governance. Technology alone is not the answer. However, since this is a technology-oriented blog, I will focus only on the state of the art in securing Java EE applications particularly in the open source space.

From a technology standpoint, I see at least four potential issues: authentication, authorization/access control, audit trail, and business process.

Spring Security (formally Acegi) has demonstrated its strength for both authentication and authorization in Spring-based portal and content/record management applications. Spring AOP (Aspect-Oriented Programming) provides an elegant and simple solution for audit trails in such systems.

JBoss jBPM is a robust BPM engine that meets the requirements for workflow and enterprise business process orchestration between applications, services, and people.

The eXtensible Access Control Markup Language (XACML) is an OASIS standard for specifying access control policies in XML. XACML is not currently widely used in content/record management systems. One explanation is that XACML has been designed to provide access control for new services such as web services in service-oriented architectures (SOA). XACML would be challenging to use for document-level security in content repositories that have a hierarchical structure (e.g. JSR 170/283 repository model) and demand sophisticated caching for scalable and rapid access to massive amounts of content.

However, my favorite XML database (eXist) has an elegant implementation of XACML for controlling access to resources such as XQuery modules and Java methods, proving once again that Open Source is ahead in terms of innovation in the software industry.

Boeing 787 Flight Control Software and Composite Fuselage Tested

Randy Tinseth, Boeing Commercial Airplanes vice-president for marketing announced on his blog that Boeing chief pilot Mike Carriker and 787 systems director Mike Sinnett successfully tested the flight-control Blockpoint 8 software code in the 787 engineering flight simulator. He wrote:

During the test, Mike and Mike demonstrated most of the operational procedures required by a flight crew - pushback and engine start at Sea-Tac airport near Seattle, taxi and takeoff, climb, cruise, simulated engine failure, descent, approach, single-engine go-around, landing, taxi and arrival at the gate at the Portland, Oregon airport.

Boeing also performed a serie of test on the composite fuselage of the B787 including "limit load", "ultimate load", and beyond 2.5 times the normal force of gravity (2.5 G). According to a Boeing press release dated 02/28/2008:

Testers observed audible indications of damage as the test progressed but the piece did not reach the level of destruction that had been anticipated.

This is a significant development because last September, Boeing announced the delay of the maiden flight of the first 787 due to flight control software issues, fastener shortage, and supply chain bottleneck. Also last year, a former Boeing engineer went public with concerns about the survivability of the 787 composite structure in case of a crash.

The maiden flight has been postponed again to June this year. First delivery to launch customer All Nippon Airways is scheduled for early 2009.

UPDATE: On April 9, 2008, Boeing has announced that 787 maiden flight has been postponed to the 4th quarter of 2008 and the first delivery for the 3rd quarter of 2009.

Speaking about the 787 globally distributed aircraft manufacturing model in an internal memo send to Boeing employees on April 21, 2008, (and obtained by the Seattle Times) Boeing CEO Jim McNerney noted:

I expect we’ll modify our approach somewhat on future programs—possibly drawing the lines in different places with regard to what we ask our partners to do, but also sharpening our tools for overseeing overall supply chain activities.

Declarative Programming Recipes

If you've been using an imperative programming language such as Java, C#, or JavaScript, then you should consider the benefits of declarative programming when designing your next application. In this blog, I will explore the benefits of the following technologies:

• XML Schema
  
• ISO Schematron
  
• XForms/XBL (XML Binding Language)
  
• XSLT 2.0
  
• XQuery
  
• Atom Syndication
  
• AtomPub
  

The basic difference between declarative programming languages and imperative programming languages such as C# and Java is that the former specify the “what” (the intent) as opposed to the “how” (the algorithm). The following are some reasons to consider this new paradigm:

• Declarative programming languages are accessible to many non-programmers
  
• It’s possible to create a solution that is completely declarative (no Java, C#, JavaScript, or AJAX code)
  
• It facilitates the Model-Driven Architecture (MDA) software design approach
  
• When the data or content is managed in XML, a declarative programming model is a superior and simpler alternative to a conventional approach based on Java Server Faces (JSF), Spring, and Hibernate. For example, you get a more robust and simpler data and business rules validation mechanism and you can forgo ORM mapping altogether. However, declarative and imperative programming can co-exist in the same solution.
  

The following are some recipes for taking advantage of this new paradigm:

• Define your data structures and types in the XML Schema.
  
• Use XML Schema's xsd:appinfo element to:
  
  
  
  • Capture metadata such such as XML to RDBMS mapping information
    
  • Capture business rules using inline ISO Schematron rules
    
  • Control the labels, alerts, and appearances of UI components such as XForms controls.
    
  
  
• Generate your UI by transforming the XML Schema into XForms with XSLT.
  
• Alternatively, store your XML schema and other XML-formatted metadata artifacts in a native XML database and use XQuery to extract and manipulate the metadata during development and at run-time.
  
• Use XBL for enabling custom UI controls for your XForms. As an example, you can integrate a rich text editor using XBL and Dojo.
  
• Use ISO Schematron for business rules validation either post form-submission or by using XSLT to inject schematron rules into the XForms directly with the xforms:bind element.
  
• Use the RESTful or AtomPub API to a native XQuery-enabled XML database (such as Exist) to CRUD (create, read, update, delete) the data.
  
• If you're using SOAP-based web services, use XForms to send SOAP requests and display SOAP responses as well.
  
• Leverage Atom syndication for pushing updates to data consumers. In general, when creating your application, think "Syndication Bus".
  

Many software architects and developers are not yet aware of the power of this new paradigm. However, I believe that rather than using only familiar development frameworks, it’s important to always evaluate emerging alternatives and select the best approach even it involves a learning curve.

Open Source and the Democratization of Knowledge

I like Open Source Software (OSS) for a number of reasons. It lowers the cost of software and allows a larger number of organizations and individuals worldwide to make use of it. OSS is also where real innovation is currently happening in the software industry. OSS companies focus their resources on the engineering process and use the web itself as a marketing and distribution channel. This allows users to freely download and try the product and decide for themselves if it’s good for them. They can get free support from the community of users or purchase commercial support for mission critical applications. The quality of product can be quickly enhanced because of the feedback that the OSS developer gets from the potentially thousands of users who download the software.

But the most important reason why I like OSS is that it enables what I call the democratization of knowledge. Complex software like operating systems, databases, ERP, CMS, and web portals all exist today in OSS. OSS is a great equalizer because anyone can access the source code to learn or discover how such complex systems are designed. They can also contribute to the code if they wish. This is unprecedented in human history. Traditionally certain groups of people have kept their technological know-how to themselves as a competitive advantage used to dominate and control markets and/or other groups of people.

Learning From My Students

For the past 10 years, I've taught a variety of technical topics in classroom settings. The subjects I've covered include XML, XML Schema, XPath, XSLT, XSL FO, SAX, DOM, XQuery, and S1000D. The participants are usually professionals who are looking to upgrade their skills. It’s gratifying to read an e-mail from a student saying how a course I’ve taught has helped them be more productive in their work or pass a certification exam.

It’s always a pleasure to share my knowledge and experience with people, but also to learn from them. When you teach a class, you must master the topic because you can't afford to always respond to students questions by saying that you will do some research and get back to them later with an answer. While preparing for the class, you need to find examples and real life scenarios to explain complex concepts to them. During the class itself, by listening to students’ questions and comments and trying to answer them, you actually discover some aspects or applications of the topic that you have not thought about before.

As the saying goes "the best way to learn is to teach".

Applicability in S1000D 3.0

S1000D has a new and improved applicability mechanism based on the concepts of Applicability Cross Reference Table (ACT), Condition Cross Reference Table (CCT), and Product Cross Reference Table (PCT).

The ACT data module declares attributes of the product that are not likely to change during its life cycle such as model, series, and serial number. Examples of product attributes for a commercial aircraft include the manufacturer serial number and aircraft registration number.

The CCT data module declares technical, operational, and environmental conditions that can affect the applicability of technical content. Examples of these conditions are: service bulletin incorporation, location of maintenance, aviation regulations, temperature, wind speed, and sandy conditions.

The PCT data module lists actual physical product instances. For each product instance, the PCT specifies the values of product attributes and conditions pertaining to the product instance.

Applicability can be specified at the data module level inside the IDSTATUS or within the content of the data module at a more granular level such as a <step1> element. The ACT and the CCT are used as look up tables to lookup the relevant product attribute or condition as well as their allowed possible values. The applicability element then specifies the correct product attribute or condition identifier from the ACT or CCT and the values to test against.

The applicability information itself can be captured in human readable format for simple cases. For more complex cases, one or more assertions are used to specify the product attribute or condition to test and the values to test against. These values can be constrained with a pattern based on regular expressions as defined by the XML Schema specification.

The new S1000D applicability mechanism supports the "effectivity" requirements of civil aviation and provides capabilities that are beyond the ATA 2200 effectivity mechanism. It also facilitates the development of applicability filtering functionalties in Interactive Electronic Publications (IETPs). However, building an authoring front end that hides its complexity (regular expressions and logical operations) to the technical authors creating the content will be the key. This is also an area where well-defined business rules should be specified and enforced using a tool such as ISO Schematron.

On our Radar in 2008

First, Happy New Year and Best Wishes for a Peaceful 2008!

Since this is my first blog this year, I will talk about what will be on our radar screen. The Java EE platform with JSF and the open source frameworks Spring and Hibernate will continue to be our preferred development platform for robust enterprise applications (ERP, portals and CMS). We like the ability to leverage third party AJAX-enabled JSF component libraries such as ICEFaces and Apache MyFaces to quickly create rich internet applications (RIA). JSF also allows us to target both the mobile and web delivery platforms simultaneously through the use of render kits. Spring brings sanity and simplicity into the development of complex Java EE applications and Hibernate allows us to develop database agnostic applications among other benefits.

Although we still believe Java EE is the way to go for complex enterprise applications, we will learn and embrace Ruby on Rails and Adobe Flex this year. Feel free to recommend your favorite books on these topics.

We'll continue to make innovative uses of ATOM syndication, AtomPub, FeedSync, and OpenSearch to solve our customers problems.

A new version of our favorite XML database (Exist) will be released soon and will improve performance significantly. Exist already has some support for AtomPub. Declarative programming using a combination of XForms, ISO Schematron, XSLT 2.0, XQuery, and Exist's RESTful API will deliver great value for those who are willing to experiment.

Finally on the Web 2.0 front, we'll be exploring the intersection of content management with social computing, mashups, user-generated content, and rich internet applications (RIA).

We look forward to another collaborative and productive year.

Extreme XML Programming

I am a proponent of agile development methodologies such as Extreme Programming and Scrum. These methodologies are based on practices such as user stories, iteration (sprint) planning, pair programming, unit test first, refactoring, continuous integration, and acceptance test. Agile programming helps create better software that is also easier to maintain.

Test-Driven Development (TDD) is a well known practice in the Java EE world with unit testing frameworks such as JUnit, EasyMock, and JMock. With TDD, you always write the test code first before you write the functional code itself. This simple principle can also be used in developing applications with XML related languages such as XSLT 2.0, XSL FO, and XQuery. Both XSLT 2.0 and XQuery 1.0 are strongly typed languages with a rich built-in function library. They can also be used to create libraries of custom functions that perform very complex business logic. Adopting a TDD methodology can increase the quality of your XSLT 2.0 or XQuery code and make it easy for you to maintain the code as you refactor and implement new requirements.

The following are my favorite unit testing frameworks:

1. Tennison-test is an XSLT Unit Test framework that allows you to write your unit tests in XML, but also allows you to automate the execution of the test as part of a build and continuous integration process based on Ant.
   
2. XMLUnit 1.1 for Java allows you to make assertions about the differences between two XML documents, the result of an XSLT transformation, the evaluation of an XPath expression, and the validity of an XML document.
   
3. Apache FOP provides a LayoutEngineTestSuite which can be used to check against the "Area Tree XML" generated by FOP's XMLRenderer.
   
4. Schema-aware XSLT 2.0 and XQuery can help root out errors early. With XML Schema 1.1's assertions and conditional type assignments, this will get even more interesting.
   
5. ISO Schematron can be used to write XPath 2.0-based assertions about the output of your XSLT or XQuery program.
   
6. xchecker is an interesting testing framework based on XML Schema, XPath 2.0, XSLT 2.0, XQuery, and Relax NG.
   
7. xSpec is a Behavior Driven Development (BDD) framework by Jeni Tennison for XSLT testing.
   

XML IDEs today provide the ability to write XSLT and XQuery code in debug mode my setting breakpoints, stepping through code one line at the time, and inspecting the value of variables. It would be nice if they could also integrate an XSLT/XQuery unit test framework a la Eclipse IDE.

Aviation Data Management in the Web 2.0 Era

I will make a presentation at the XML 2007 Conference in Boston on December 5. The title of my presentation is: RESTful IDEAS. IDEAS stands for Integrated Documentation Environment for Aircraft Support. Based on AtomPub and OpenSearch, the IDEAS Framework enables federated searches of technical content and updates via web feeds. This presentation is essentially about how Web 2.0 innovations can be leveraged to create a cost-effective, efficient, and massively scalable environment for aggregating and publishing up-to-date technical documentation to end users in the aviation industry.

So what exactly is Web 2.0? Does it mean anything at all? For me Web 2.0 represents new web functionalities such as:

• Content aggregation and syndication using technologies like RSS and Atom
  
• Social networking (e.g. Linkedin and Facebook)
  
• User generated content with videos, pictures, blogs, wikis, and podcasts (e.g. YouTube and Flickr)
  
• Mashups or the ability to merge data from different sources (e.g. Google Maps and Yahoo Pipes)
  
• Rich Internet Applications (RIA) using user interface technologies such as Flex and AJAX
  
• Web Services and web-based APIs particularly those using RESTful protocols such as AtomPub
  

Those are the types of functionalities that today's web users expect from web applications, and the consumers of aviation technical documents are no exception.

Atom and AtomPub are already playing a fundamental role in the Web 2.0 world. The Google Data API which also includes Google Documents is based on AtomPub and OpenSearch. This week, a group of social networking sites including hi5, LinkedIn, MySpace, Ning, Orkut, and XING released OpenSocial which is a common set of APIs for developing social applications across multiple websites. OpenSocial is also based on AtomPub.

So how can the aviation industry embrace and extend Web 2.0 innovations to solve the challenge of aggregating and presenting up-to-date technical data from multiple content sources? I will be sharing my thoughts.

See you in Boston!

S1000D and SCORM Integration

I gave a presentation yesterday on S1000D and SCORM integration at the Doctrain 2007 conference in Lowell, MA. The main goal of integrating these two specifications is to reduce product life cycle costs and eliminate redundancies by streamlining business processes across the documentation and training functions. I noted that there are many opportunities for data reuse at every phase of the product life cycle including: concept, design, manufacturing, assembly, testing, delivery, and support. Documentation and training belong to the support phase and are often the only departments where content is captured in XML. This is due to the complexity and cost of current specialist XML authoring tools. XForms will allow knowledge workers to contribute knowledge assets in XML at every phase of the product lifecycle with a simple web form. I also believe that Office Open XML (OOXML) offers the opportunity to extract some value out of MS Office documents by exposing their contents to XML processing languages and tools such as XSLT 2.0 and XQuery.

Engineering data should be the trusted source of data for both publications and training. For example, product model data and engineering drawings can be used to create simulation for training and manufacturing assembly instructions can be used to create installation procedures for publications. Any data reuse strategy should look beyond training and publication to identify ways to reuse data and streamline processes across the entire product lifecycle.

The integration of SCORM and S1000D presents management and technical challenges. Since training and documentation are often two separate functions within the enterprise, the integration can have an impact on budget, processes, roles, and the organizational structure. For example, the cross-functional integrated project team is a good approach. The success of the integration will also require top leadership commitment and support. It is also important to address technical challenges such as the integration of existing content management systems (CMS) and learning management systems (LMS).

The first technical approach is to create dual purpose data modules (DMs). I highlighted the importance of clearly defining and documenting business rules, particularly for dual purpose data modules. The business rules should specify among other things the appropriate level of granularity and language style (e.g. Simplified English). These business rules should be validated with technologies such as ISO Schematron and Simplified English Checkers. Since S1000D is weak on learning content metadata, the IEEE Learning Object Metadata (LOM) specification should be used to add learning object metadata to the S1000D dual purpose DMs. It is also possible to package all training data modules as a training publication module (PM). XSLT is then used to transform the S1000D DM into SCORM sharable content objects (SCOs) and learning assets. The S1000D metadata (IDSTATUS) shall be retained in the result SCOs to facilitate product applicability filtering when the SCOs are presented to the learners with an LMS. The S1000D PM can be used to generate the SCORM manifest as well. The dual purpose S1000D DM approach does not always support complex learning interactions and good instructional design principles.

The second approach is to give complete freedom to the instructional designer to design an effective learning experience. All elements in the S1000D data modules that are reusable in SCOs are assigned a unique ID. Examples are: paragraphs, steps, warning, cautions, notes, tables, etc. This can be done automatically using the XSLT generate-id() function. The instructional designer then searches the CSDB to find and display relevant DMs. She can then use XInclude/XPointer to include reusable elements from the DM into the SCO. When this is done, the SCO is automatically updated when the DM is updated.

Future versions of the S1000D specification will incorporate change proposal forms (CPFs) that will facilitate the integration of SCORM and S1000D content.

A copy of my presentation is available here.