National Information Exchange Model
October 1, 2019
|Organization||NIEM Executive Steering Council|
|Base standards||XML, XML Schema|
The National Information Exchange Model (NIEM) (// neem) is an XML-based information exchange framework from the United States. NIEM represents a collaborative partnership of agencies and organizations across all levels of government (federal, state, tribal, and local) and with private industry. The purpose of this partnership is to effectively and efficiently share critical information at key decision points throughout the whole of the justice, public safety, emergency and disaster management, intelligence, and homeland security enterprise. NIEM is designed to develop, disseminate, and support enterprise-wide information exchange standards and processes that will enable jurisdictions to automate information sharing.
NIEM is an outgrowth of the United States Department of Justice's Global Justice XML Data Model (GJXDM) project. NIEM is now being expanded to include other federal and state agencies such as the Office of the Director of National Intelligence, United States Department of Defense, Federal Bureau of Investigation, Texas, Florida, New York, Pennsylvania, and others.
NIEM is designed to facilitate the creation of automated enterprise-wide information exchanges which can be uniformly developed, centrally maintained, quickly identified and discovered, and efficiently reused.
The following key concepts are essential to understanding the purpose, architecture, processes, and other capabilities of NIEM, as well as to establish a common knowledge base with which to develop the ability to use NIEM effectively.
Data Components. The fundamental building block of NIEM is the data component. Data components are the basic business data elements that represent real-world objects and concepts. Information exchanged between agencies can be broken down into individual components – for example, information about people, places, material things, and events. Components that are frequently and uniformly used in practice are specified in NIEM and can then be reused by practitioners for information exchanges, regardless of the nature of their business or the operational context of their exchanges, provided they are semantically consistent.
Information Exchange Package Documentation. The information that is commonly or universally exchanged between participating domains can be organized into information exchange packages (IEPs) in the form of XML Schemas. An example of this collection of information is data associated with an arrest. The data to be exchanged includes not only descriptive and personal identification data regarding the individual arrested (i.e., the person component described above) but also information about the person's alleged offense, the location of the offense, the arresting officer, etc. The IEP represents a set of data that is actually transmitted between agencies for a specific business purpose (e.g., initiating a charging document by the local prosecutor). It includes the actual XML instance that delivers the payload or information. Additional information regarding this specific exchange can be further documented in the form of an information exchange package documentation (IEPD), which also contains data describing the structure, content, and other artifacts of the information exchange. An IEPD supports a specific set of business requirements in an operational setting.
NIEM Core. Data components within an information exchange that are universally shared and understood among all (or almost all) domains are identified as universal components (e.g., person, address, and organization). To become a universal component, consensus by all domains is needed on the semantics and structure of the component. The set of NIEM universal components is stable (once established) and relatively small.
Domains. For purposes of NIEM, a domain refers to a business enterprise broadly reflecting the agencies, units of government, operational functions, services, and information systems which are organized or affiliated to meet common objectives. NIEM domains are organized to facilitate governance, and each has some measure of persistency. Each domain traditionally includes a cohesive group of data stewards who are subject matter experts (SMEs), have some level of authority within the domains they represent, and participate in the processes related to harmonizing conflicts and resolving data component ambiguities.
Communities of Interest. Communities of interest (COIs) are collaborative groups of users who exchange information in pursuit of shared goals, interests, missions, or business processes and who therefore must have a shared vocabulary for the information they exchange. COIs reuse data components and artifacts found in NIEM to document their information exchanges. One or more COIs can coordinate to develop new domain content as they identify gaps in the data components needed for documenting information exchanges.
NIEM Conformance. There are NIEM conformance rules that serve as guidelines for agencies utilizing NIEM to implement their information sharing exchanges. Grantees developing inter-agency XML-based exchanges must comply with the special condition language contained in the grant, and follow the associated NIEM implementation guidelines.
NIEM Program Management Office
The NIEM Program Management Office (PMO) operates to:
- Bring stakeholders, agencies, and the domains and COIs that they represent together to identify information sharing requirements in daily operational and emergency situations;
- Develop information sharing standards, a common lexicon, and an online repository of information exchange package documentation and data components that support information sharing;
- Provide technical tools, processes, and methodologies to support the analysis, development, discovery, dissemination, and reuse of exchange standards and documents; and
- Provide training, technical assistance, communication, outreach, and implementation support services for NIEM-based information sharing.
Training and other technical resources
NIEM is a continually evolving program, and new agencies and COIs are joining the effort all the time. As new stakeholders come on board, they need to receive information to gain understanding and knowledge of the core capabilities of NIEM and how to engage in NIE information exchanges. NIEM.gov provides training materials, such as briefings and process-related documentation, as well as other resources, such as the National Information Sharing Standards Help Desk and Knowledge Base. Training provides the knowledge and know-how stakeholders need to use the tools and other capabilities provided by NIEM. NIEM tools and training opportunities are further described below. Other training materials, such as executive briefings, marketing material, and briefings for conferences and workshops, are offered from time to time and are tailored depending on the audience. A selection of online materials are also available both from the main NIEM.gov web site and resources such as Youtube.com (search on NIEM training).
The NIEM Web site serves as a primary means by which NIEM can provide the latest documentation and downloads to those interested in NIEM. It also serves as a starting point for those wishing to contact NIEM staff with questions, support, and information requests. As related projects, tools, and support resources develop around NIEM, the Web site will expand as the hub for these supplemental resources.
The NIEM reference schemas are a set of interrelated schemas that define NIEM data components. Each schema defines its own target namespace. Schemas in the reference set may import one another by namespace in order to use (or reuse) components they define. In general, domain reference schemas import schemas from the Core. The NIEM reference schema set represents the full set of data components in NIEM.
The following kinds of XML schemas are associated with the NIEM architecture:
- NIEM reference schemas: Schemas containing content created or approved by the NIEM steering committees are periodically released in schema distributions.
- Subset Schema: a NIEM-conformant schema, containing only the parts of the reference schemas needed to support a particular exchange.
- Support schemas: NIEM includes three special schemas, appinfo, structures and proxy, for annotating and structuring NIEM-conformant schemas.
- Extension Schema: a NIEM-conformant schema which adds domain- or application-specific content to the base NIEM model.
- Exchange Schema: a NIEM-conformant schema which specifies a document in a particular exchange.
- Constraint Schema: a NIEM-conformant schema which adds additional constraints to NIEM-conformant instances, but which is assumed to validate in concert with existing NIEM-conformant or subset schemas. A constraint schema need not validate constraints that are applied by other schemas.
- Codelist Schemas: a NIEM-conformant schema which provides a list of acceptable values that a data element will be constrained to.
The only mandatory schemas for validation are the NIEM reference schemas or a correct subset, however the IEPD specification requires that an IEPD include an exchange schema (along with the reference schemas or subsets) to be considered a complete IEPD. The NIEM schemas may import additional schemas, such as code table schemas, as needed. An optional extension schema may be used to add extended types and properties for components not contained in NIEM, but which are needed for the exchange.
NIEM Naming and Design Rules (NDR)
The naming and design rules for NIEM are documented in the NIEM NDR, which specifies the data model, XML components, and XML data for use with NIEM and provides a basis for NIEM conformance. The current version is NDR v4.0, which was released on July 13, 2017).
NIEM is based on several concepts from the International Organisation for Standardisation (ISO) 11179, which provides guidelines for the naming and definition of data elements, as well as information about the metadata captured about data elements. Part 5 of the ISO 11179 standard establishes a methodology for naming items in data dictionaries.
The ISO 11179-based NIEM NDR naming convention uses object class, property, and representation terms to constitute a multiple-part name as shown in the figure below:
Object Class Term: Represents the object to which the property is applicable. In NIEM, we interpret that object to be the real-world object. (An object class refers to a group of objects that share the same attributes, operations, methods, relationships, and semantics.)
Property Term: Identifies the property that the data element represents (e.g., last name, expiration date, height, total).
Representation Term: Describes the form of the data represented. This term is taken from a list of electronic business XML (ebXML) representation terms, including amount, code, date, time, graphic, identifier, indicator, measure, name, percent, picture, quantity, rate, time, and numeric.
Qualifier Term: The object class and property terms can have qualifiers; i.e., a word or words that help define and differentiate the element name.
NIEM modeling and schema concepts
The NIEM technical modeling and schema concepts and mechanisms which support building new data components that meet specific requirements and reusing existing NIEM data components are briefly described below. More detail can be found in the NIEM User Guide and the NIEM NDR.
Data elements, classes, types, and properties
The NIEM data model uses concepts originating from object-oriented programming (OOP). OOP defines a class as a specific entity in the data model, which may represent a real-world object but may also represent any conceptual object, such as relationships and messages. An object's properties are said to describe the object. When the NIEM XML Schemas are generated from the NIEM data model, data model classes are represented as XML Schema types, and data model properties are represented as XML elements and attributes.
There are two approaches for extending the NIEM data model for use in information exchange schemas and documents.
- Creating new NIEM types to represent new concepts
- Adding new data to existing NIEM types, to extend existing concepts
Specialization with inheritance
Specialization is used when a base object class (type) contains or can be subcategorized into a more specific subclass. When this can be done, the subclass derived from the base class inherits the properties of the more general base or parent class. This mechanism is used to share or reuse properties between the general data component and its specialization. For example, a vehicle type (or class) is identified as a data component with properties of vehicle identification number (VIN), make, and model. Truck type (or class) is a specialization of vehicle and thus inherits the vehicle's properties but also has its own characteristic properties, such as truck bed length. Specialization is time independent and is generally used only when the base class and subclass always exist.
A role is a special type which represents a particular function, purpose, context, or activity for an entity. Roles are generally time dependent and, therefore, temporary. A new type can be created for a role when the role has specific data associated with it and its own life cycle. A role type has a property, RoleOf, which indicates what object is assuming this role. A single entity may assume multiple roles. For example, many different entities may assume the role of a weapon. Therefore, if a vehicle is used as a weapon (to attempt to injure or kill a person), then an instance of WeaponType would contain the property, RoleOf, which references the vehicle instance used as the weapon. The WeaponType (the role) might also contain properties that describe the persons and activities involved, dates and times of involvement, and how the entity was used as a weapon.
An association type is an object that represents a relationship between data components. For example, two PersonType instances, Abigail and Bob, could be referenced by a MarriageAssociationType to represent the fact they are married. The MarriageAssociationType could contain its own properties, such as date of marriage, number of children, date of divorce, death of one spouse, etc.
Augmentation of a NIEM data type allows the addition of domain- or model-specific information to the concept embodied in the NIEM type, without creating a new NIEM type. It would be impractical and unwieldy to include all possible domain model-specific properties in NIEM Core schemas for general use. Instead, domain modelers need to be able define data for their use, independently from common definitions. Furthermore, that data needs to be applicable to the NIEM data object itself, and reusable in multiple exchanges. The augmentation approach built into NIEM utilizes XML Schema constructs to reuse the existing XML schema representations for the data model, by allowing them to be augmented with the new information.
NIEM uses several techniques from XML Schema to allow as-needed element substitutions for pre-existing NIEM properties and into pre-existing NIEM types. Element substitution techniques allow the substitution of new XML Schema elements, representing derived NIEM properties that can be used where the parent properties are expected.
There are three XML Schema techniques that support the NIEM use of element substitutions:
- use of substitution groups
- creation of abstract, type-less elements, and
- use of abstract elements in reference schemas.
Adapting external standards
In addition to adding new NIEM types and properties to NIEM, it is possible to adapt existing external (non-NIEM) namespaces for use in the NIEM framework. This allows the use of external standards within NIEM IEPDs, without requiring that the external standards themselves be NIEM-conformant. The intent here is to allow use of external standard components exactly as they were defined.
Metadata, or data about data, defines information that supports the actual content of XML instances. The metadata feature provides a mechanism for attaching structured properties that describe the pedigree or source (when reported, who reported, how reliable, etc.) of instance data to any data component of the model (type or object, property, association, role, or augmentation) in any namespace. It allows sets of metadata to be extended with additional properties for local requirements and enables metadata properties to be repeated.
EIEM dictionary concepts
NIEM has introduced the concept of Enterprise Information Exchange Model (EIEM) dictionaries to allow NIEM implementers to position local collections of components specific to their own enterprise. Typically these may be drawn from an Enterprise Data Model (EDM) and then exported and refactored as XML components that conform to the NIEM NDR. See Tools section below for information on creating EIEM dictionaries.
NIEM provides a reference set of tools freely available with each NIEM release. The tools implement all of the structural and content features of the release, including the NIEM NDR. NIEM's well-defined interfaces and output products also support the development of independent third-party tools. Example supporting third party tools are listed below.
NIEM XML data dictionary spreadsheet
The NIEM XML data dictionary spreadsheet is a tangible representation of the entire NIEM data dictionary. It includes all of the element names that are organized hierarchically under core data components (person, property, organization, etc.) with hyperlinks to related elements. This spreadsheet also provides information on the type of data being represented (date, integer, string, etc.) and a precise, context-rich definition of each dictionary component.
Information exchanges mapping tool
The Information Exchanges Mapping Tool allows the user to specify metadata and upload XMI domain models associated with NIEM IEPDs, map components within domain models to NIEM components, and generate artifacts based on mappings, including mapping reports, wantlist, exchange schemas, extension schemas, and subset schemas.
Schema Subset Generation Tool
The Schema Subset Generation Tool (SSGT) enables users to search through the NIEM data model and build a NIEM subset. The NIEM data model covers several domains and typically not all of the data model is usable in an exchange. It is useful to make a schema subset of NIEM components to use in an exchange. It helps to limit the scope of developing an IEPD and can be built to the specific requirements of an exchange. With this tool users can:
- Search the data model from the Search Page.
- Navigate through the data model.
- Download already generated subsets from each domain.
- Build a schema subset containing the components required by their exchange.
- Save a list of components required for a subset.
- Upload a list of requested components to continue to edit it.
- Download a schema subset based on a list of required components.
- Create IEPD with current subset.
- Change the NIEM release used for building a schema subset.
The IEPD tool enables the user to upload or enter the artifacts required for an IEPD (schemas, documentation, and metadata) and assembles into a package according to the IEPD specification. It can also validate that minimum artifacts and metadata are present. The user creates an account and is granted a work space ("My IEPDs"). Inside this work space, the user can upload the artifacts to construct any number of IEPDs (complete or partial); share them with other account holders; or search, discover, and download IEPDs that other account holders have marked for sharing.
A Model Package Description Specification specifies the artifacts to be included in an IEPD or other Information Exchange Model (IEM).
Additionally see Third Party NIEM tools section below for IEPD tools that can automate much of the IEPD artifact generation. 3rd Party IEPD tool support includes automatic generation of NIEM cross-reference, wantlist.xml, exchange and subset schema, XML examples, rules documentation and NDR evaluation.
IEPDs which have been published to the IEPD Clearinghouse are available for reuse by other organizations.
This section needs to be updated.October 2019)(
NIEM 2.1 was released on September 28, 2009 and includes a number of enhancements:
- Three new domains—Maritime; Family Services; and Chemical, Biological, Radiological, Nuclear (CBRN)
- Updates to the Justice, Infrastructure Protection, and Emergency Management domains
- Harmonization to reduce overlapping and duplicate data elements between domains
- Completion of plain-English definitions for all components
- 35% more content than NIEM 2.0
As of August 2012[update], development of NIEM 3.0 is in progress and is planned for release in fall 2013. NIEM 3.0 will include core updates, domain content updates, technical architecture enhancements, and updated tools.
- NIEM 4.2 Production Release – October 1, 2019
- NIEM 4.1 Production Release – August 1, 2018
- NIEM 4.0 Production Release – June 28, 2017
- NIEM 3.1 Production Release – May 29, 2015
- NIEM 3.1 Release Candidate – April 29, 2015
- NIEM 3.1 Beta Release – Mar 23, 2015
- NIEM 3.0 Production Release – October 21, 2013
- NIEM 3.0 Release Candidate – September 4, 2013
- NIEM 3.0 Beta – June 25, 2013
- NIEM 3.0 Alpha 2 – April 19, 2013
- NIEM 2.1 Production Release – September 28, 2009
- NIEM 2.1 Release Candidate 1 – September 1, 2009
- NIEM 2.1 Beta 1 – July 29, 2009
- NIEM 2.0 Production Release – July 31, 2007
- NIEM 2.0 Release Candidate 2 – June 2007
- NIEM 1.0 Production Release – November 1, 2006
- NIEM 1.0 Release Candidate 1 – September 30, 2006
- NIEM 1.0 Beta 3 – September 11, 2006
- NIEM 1.0 Beta 2 – August 8, 2006
- NIEM 1.0 Beta 1 – June 30, 2006
- NIEM 0.3 – April 12, 2006
- NIEM 0.2.1 – February 24, 2006
- NIEM 0.2 – December 23, 2005
- NIEM 0.1 – October 11, 2005
- Rules for NIEM Conformance
- Global Justice XML Data Model (GJXDM)
- ISO/IEC 11179
- Metadata publishing
- Metadata registry
- Universal Core (UCore)
- "NIEM 4.2 Summary". niem.gov. Retrieved October 1, 2019.
- "NIEM Newsletter". niem.gov. July 2009. Retrieved 2011-09-01.[dead link]
- "NIEM 3.0". niem.gov. July 9, 2012. Archived from the original on 2014-01-04. Retrieved 2013-08-01.
- "Coming Soon NIEM 3.0". niem.gov. Archived from the original on 2013-07-29. Retrieved 2013-08-01.
- "NIEM Releases". niem.github.io. Retrieved October 1, 2019.
- "NIEM Version 4.2 is Now Available". niem.gov. Retrieved October 1, 2019.
- "NIEM Version 4.1 is Now Available". niem.gov. Retrieved August 2, 2018.
- "NIEM version 3.1 is now available". Archived from the original on 2016-07-22. Retrieved 2016-05-26.