 |
DDI Foundation Tools Program: Roadmap
The purpose of the DDI-FTP Roadmap is to provide anyone interested in the program with a description of the planned components and general guidelines for their development. This document outlines the overall framework, describes the various components and their relationships, recommends the technologies and tools to be used for the implementation, and contains guidelines and best practices for software developers. The Roadmap is used to track the development progress and priorities.
The DDI-FTP Framework and the product Road Map is available for review in the document below. The DDI-FTP follows the general recommendations provided by the Open Data Foundation for the development and operation of open source tools for managing social, behavioral and economic data and metadata.
- "DDI-FTP: Framework and Road Map", Feb 2008
- "Managing social, behavioral and economic data and metadata: Guidelines for Tools Development and Recommendations for Operating Environment", Open Data Foundation, Feb 2008
- "DDI Editor Framework Specifications [DRAFT]", DDI Foundation Tools Program, Oct 2008
Framework overview
The following diagram illustrates the main components of the framework and road map. Click on thumbnail to see a larger version.
Products Groups
| Name | Description |
|---|---|
| Stand-Alone Products | Desktop-based application for data/metadata managers and end-users |
| Web-Based Products | End-user applications or light administrative tools accessible over the Internet (or Intranet) using a standard Web browser |
| Web Services | Web service oriented architecture for the management of DDI and related resources. Provides access to metadata and data through commonly used Web services protocols such as SOAP, REST, and others. |
| DDI 1/2.x Utility Library | Function library to manage DDI 1/2.x documents |
| DDI 3.0 Utilities | Libraries to manage DDI 3.0 documents |
| DDI 3.0 URN Library | Library to facilitate the management of URNs and Identifiers in DDI 3.0 |
| DDI 3.0 Scheme Library | Tools to facilitate the management of DDI 3.0 Schemes |
| DDI 3.0 Grouping and Comparability Library | Library to facilitate the grouping of DDI 3.0 entities and the use of the comparability features. |
| DDI 3.0 Physical Product Libraries | Libraries to facilitate the management of record layouts, data cubes and physical data fie instances. |
| Internationalization / Localization Utilities | Libraries to support the management of multi-lingual metadata |
| XSL Transformations | Collection of XSL Transformations for DDI |
| DDI Conversion Library | Library to facilitate the conversion of DDI metadata between specifications |
| XMLBeans | Java classes for the management of DDI documents based on the Apache XML Beans technology Provides object classes/methods to manage DDI 1/2.x and 3.0 metadata. See http://xmlbeans.apache.org/ for further information. |
| Data Management Libraries | Tools to manage statistical data files |
| Resource Management Libraries | Tools to manage digital resources such as documents, multimedia and other files. |
| File Storage System | Tools to facilitate the storage and retrieval of files |
| Metadata Storage System | Libraries to facilitate the storage and retrieval of metadata |
| Security | Tools to control access to resources and provide authentication, authorization and encryption of data and metadata |
| General Utilities | Collection of various system utilities |
High Priority Products
| Name | Description | Group | Status | Other |
|---|---|---|---|---|
| DDI Converter | A tool to convert DDI documents between different versions of the DDI and to other metadata specifications. The initial focus on this tool is on the upgrade of DDI 1/2.x documents into 3.0 as well as the downgrade of a 3.0 document into earlier versions. | Stand-Alone Products | not started | medium/10
|
| DDI 3.0 URN Resolution Services | A service to support the conversion of a URN into a URL | Web Services | not started | high/10
|
| ID Generator | Function(s) to generate DDI 3.0 unique IDs. | DDI 3.0 URN Library | initiated | low/10
|
| URN Generator | Functions to generate a DDI 3.0 URN based on an element identifier. | DDI 3.0 URN Library | beta | low/10
|
| DDI 3.0 XMLBeans | Apache XMLBeans for DDI 3.0 | XMLBeans | completed | low/10
|
| Prototype Editors | Simple DDI editors to be used for training purposes and testing of the DDI-FTP framework. The focus will be on viewing and editing subset of the DDI 3.0 (or 1.2/x) | Stand-Alone Products | in progress | medium/9
|
| DDI 3.0 Parser / Validator | DDI 3.0 specific XML parser that performs schema and second level validation of DDI 3.0 documents. | DDI 3.0 Utilities | in progress | high/9
|
| URN Resolver | Functions to convert a DDI 3.0 URN into a URL | DDI 3.0 URN Library | not started | medium/9
|
| Scheme Manager | An abstract class to define methods to be implemented by all scheme managers | DDI 3.0 Scheme Library | not started | easy/9
|
| DDI 3.0 XSLT Library | Library of XSL Transformations for DDI 3.0/x | XSL Transformations | in progress | medium/8
|
| DDI 2.0 to 3.0 Upgrade Functions | Functions that take a DDI 1/2.x document and convert to a DDI 3.0 document. The process should be as automated as possible but will most of the time require user interaction to provide information and make mapping decision. | DDI Conversion Library | not started | medium/8
|
| DDI 2.0 to 3.0 Upgrade Guide | Guide for the preparation of DDI 1/2.x document for upgrade to 3.0 | DDI Conversion Library | not started | medium/8
|
Products List
Stand-Alone ProductsDesktop-based application for data/metadata managers and end-users
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| DDI Converter | A tool to convert DDI documents between different versions of the DDI and to other metadata specifications. The initial focus on this tool is on the upgrade of DDI 1/2.x documents into 3.0 as well as the downgrade of a 3.0 document into earlier versions. | not started | medium/10
|
| Prototype Editors | Simple DDI editors to be used for training purposes and testing of the DDI-FTP framework. The focus will be on viewing and editing subset of the DDI 3.0 (or 1.2/x) | in progress | medium/9
|
| Prototype Grouping Tools | A tool that imports details of question, variable, category, code, universe, and concept and then display the items selected for comparison side by side. The goal would be to be able to view the source and target and then enter the comparison information in the DDI comparison structure. | not started | medium/6
|
| DExT Exchange Tools | DExT Exchange Tools is a collaborative project between the UK Data Archive and the Open Data Foundation for the development of data archiving and conversion tools. Supported by the UKDA Data Exchange Tools (DExT) initiative, the project aims at the development of open source utilities for the conversion of quantitative data into a standard archiving format and export to common statistical packages or database systems. This initial product is a proof of concept and focuses on SPSS as the input data format with data export capabilities to ASCII, SAS, Stata, and SPSS. The application is based on the DDI specification and supports metadata export to DDI 2.0 and DDI 3.0. | completed | high/7
|
| StatsProgs2DDI | Conversion tool from statistical package formats to DDI Version 3.0. Currently the tool includes a SPSS converter; further planned formats are SAS, and Stata. This component is contributed by GESIS/ZUMA. Requires SPSS 14+. | beta | medium/7
|
Web-Based ProductsEnd-user applications or light administrative tools accessible over the Internet (or Intranet) using a standard Web browser
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| DDI Survey Catalog | End-user application to browse catalog of surveys and their related metadata | not started | medium/7
|
| WebDExT for SPSS | A web based application to export DDI 2.0 or 3.0 metadata from SPSS files and to generate setup files for SAS, SPSS and Stata. | not started | medium/7
|
Web ServicesWeb service oriented architecture for the management of DDI and related resources.
Provides access to metadata and data through commonly used Web services protocols such as SOAP, REST, and others.
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| DDI 3.0 Registry | A service to support the registration, search and retrieval of DDI 3.0 entities | not started | high/7
|
| DDI 3.0 URN Resolution Services | A service to support the conversion of a URN into a URL | not started | high/10
|
DDI 1/2.x Utility LibraryFunction library to manage DDI 1/2.x documents
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| DTD2Schema | Utility function to convert a DTD-based DDI 1/2.x document into a XMLSchema based version. The library performs validation and reports any problems (along with potential solutions) to the user. | not started | low/5
|
| DDI 1.2/x Flavor Detector | Utility function(s) that attempt to detect the metadata producer and/or software used to prepare the DDI document. This information is useful to other components in order to gain knowledge on the expected content of the DDI Document. Could also be used to assign a DDI Profile when upgrading to DDI 3.0. | not started | low/5
|
| DDI 3.0 Compatibility Test | Function to assess the degree of compatibility of a DDI 1/2.x document with the DDI 3.0 specification. | not started | low/7
|
DDI 3.0 UtilitiesLibraries to manage DDI 3.0 documents
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| DDI 3.0 Parser / Validator | DDI 3.0 specific XML parser that performs schema and second level validation of DDI 3.0 documents. | in progress | high/9
|
| DDI 3.0 Gatherer | Method(s) to process a DDI 3.0 document containing external references and collect the various pieces of information to recompose a local instance of the document with all references replaced with their actual content. | not started | high/6
|
| CodeBook Manager | Class to facilitate the management of DDI 3.0 variables, codes, categories, formats, logical record, physical data product and physical instance from the classic perspective of a code book. This will provide applications (and users) with a virtual "view" of the DDI 3.0 that replicates the typical approach used by statistical packages. | initiated | medium/5
|
DDI 3.0 URN LibraryLibrary to facilitate the management of URNs and Identifiers in DDI 3.0
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| ID Generator | Function(s) to generate DDI 3.0 unique IDs. | initiated | low/10
|
| URN Generator | Functions to generate a DDI 3.0 URN based on an element identifier. | beta | low/10
|
| URN Resolver | Functions to convert a DDI 3.0 URN into a URL | not started | medium/9
|
DDI 3.0 Scheme LibraryTools to facilitate the management of DDI 3.0 Schemes
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| Scheme Manager | An abstract class to define methods to be implemented by all scheme managers | not started | easy/9
|
| Concept Manager | Class to facilitate the management of concepts | not started | easy/7
|
| Universe Manager | Class to facilitate the management of universes | not started | easy/7
|
| Question Manager | Class to facilitate the management of questions | not started | easy/7
|
| Classification Manager | Class to facilitate the management of categories and codes | not started | easy/7
|
| Geography Manager | Class to facilitate the management of geography structure and locations | not started | easy/7
|
| Variable Manager | Class to facilitate the management of variables | not started | easy/7
|
| Organizations Manager | Class to facilitate the management of organizations and individuals | not started | easy/7
|
DDI 3.0 Grouping and Comparability LibraryLibrary to facilitate the grouping of DDI 3.0 entities and the use of the comparability features.
| |||
| No product defined in this group |
DDI 3.0 Physical Product LibrariesLibraries to facilitate the management of record layouts, data cubes and physical data fie instances.
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| ASCII File Metadata Manager | Library to facilitate the management of ASCII files metadata | not started | medium/7
|
| Proprietary Files Metadata Manager | Library to facilitate the management of proprietary files metadata | not started | medium/7
|
| Cubes Metadata Manager | Library to facilitate the management of data cubes | not started | medium/7
|
Internationalization / Localization UtilitiesLibraries to support the management of multi-lingual metadata
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| XML Spell Checker | Tool to check spelling in XML metadata | not started | medium/5
|
XSL TransformationsCollection of XSL Transformations for DDI
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| DDI 1/2.x XSLT Library | Library of XSL Transformations for DDI 1.2/x | in progress | medium/5
|
| DDI 3.0 XSLT Library | Library of XSL Transformations for DDI 3.0/x | in progress | medium/8
|
| XSLT Utility Library | Library of XSL Transformations for other metadata specifications | in progress | medium/5
|
DDI Conversion LibraryLibrary to facilitate the conversion of DDI metadata between specifications
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| DDI 2.0 to 3.0 Upgrade Functions | Functions that take a DDI 1/2.x document and convert to a DDI 3.0 document. The process should be as automated as possible but will most of the time require user interaction to provide information and make mapping decision. | not started | medium/8
|
| DDI 3.0 to 2.0 Downgrade Functions | Functions that convert a DDI 3.0 document into one of the official version of DDI 1/2.x. | not started | medium/6
|
| DDI 2.0 to 3.0 Upgrade Guide | Guide for the preparation of DDI 1/2.x document for upgrade to 3.0 | not started | medium/8
|
| DDI1/2.x Mappers | Collection of functions and XSL transformations to convert DDI 1/2.x to/from various metadata specification such as Dublin Core, MARC, etc. | not started | medium/3
|
| DDI3 Mappers | Collection of functions and XSL transformations to convert DDI 3.0 to/from various metadata specification such as Dublin Core, MARC, SDMX, etc. | not started | medium/4
|
| Boundary Converter | Library to facilitate the conversion of geographic boundary files (E00,ESRI shape files, etc.) into the DDI format. The purpose is not to convert detailed boundaries into the DDI format but rather to generate bounding boxes or a reduced set of points. | not started | medium/5
|
XMLBeansJava classes for the management of DDI documents based on the Apache XML Beans technology
Provides object classes/methods to manage DDI 1/2.x and 3.0 metadata.
See http://xmlbeans.apache.org/ for further information.
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| DDI 1.0 XMLBeans | Apache XMLBeans for DDI 1.0 | completed | low/5
|
| DDI 1.1 XMLBeans | Apache XMLBeans for DDI 1.1 | completed | low/5
|
| DDI 1.2 XMLBeans | Apache XMLBeans for DDI 1.2 | completed | low/5
|
| DDI 1.2.1 XMLBeans | Apache XMLBeans for DDI 1.2.1 | completed | low/5
|
| DDI 1.2.2 XMLBeans | Apache XMLBeans for DDI 1.2.2 | completed | low/5
|
| DDI 1.3 XMLBeans | Apache XMLBeans for DDI 1.3 | completed | low/5
|
| DDI 2.0 XMLBeans | Apache XMLBeans for DDI 2.0 | completed | low/5
|
| DDI 2.1 XMLBeans | Apache XMLBeans for DDI 2.1 | completed | low/5
|
| DDI 3.0 XMLBeans | Apache XMLBeans for DDI 3.0 | completed | low/10
|
| SDMX 1.0 XMLBeans | Apache XMLBeans for SDMX 1.0 | beta | low/2
|
| SDMX 2.0 XMLBeans | Apache XMLBeans for SDMX 2.0 | beta | low/3
|
Data Management LibrariesTools to manage statistical data files
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| SPSS Reader | Java package to read SPSS data files and (1) export the data to various ASCII formats and (2) generate metadata in DDI 2.0 and DDI 3.0 formats. This component is contributed by the UK Data Archive DExT initiative and the Open Data Foundation. | beta | easy/7
|
| Stata Reader | Java package to read Stata data files, generate DDI 2.0 and 3.0 metadata and optionally export data in ASCII formats | initiated | easy/7
|
| SAS Reader | Java package to read SAS data files, generate DDI 2.0 and 3.0 metadata and optionally export data in ASCII formats | not started | medium/7
|
| Setup File Generator | Functions using DDI metadata to generate setup files for the import of ASCII data into SPSS, Stata, SAS, SQL and other software packages | in progress | medium/5
|
Resource Management LibrariesTools to manage digital resources such as documents, multimedia and other files.
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| Dublin Core Library | Tool to facilitate the compilation of Dublin Core metadata | not started | easy/3
|
| Citation Management Library | Library to facilitate the shared management of paper citation | initiated | easy/3
|
File Storage SystemTools to facilitate the storage and retrieval of files
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| File Storage System | Tool to add, update, delete digital resources in a file repository | not started | medium/5
|
| File Query Engine | Tool to search and retrieve digital resources from a file repository | not started | medium/5
|
Metadata Storage SystemLibraries to facilitate the storage and retrieval of metadata
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| DDI 3.0 Metadata Store | Library to add, update, delete DDI 3.0 documents in a XML repository | initiated | high/7
|
| DDI 3.0 Metadata Query Engine | Library to search and retrieve DDI 3.0 documents from an XML repository | initiated | high/7
|
| DDI 1/2.x Metadata Store | Library to add, update, delete DDI 3.0 documents in a XML repository | initiated | medium/5
|
| DDI 1/2.x Metadata Query Engine | Library to search and retrieve DDI 3.0 documents from an XML repository | initiated | medium/5
|
SecurityTools to control access to resources and provide authentication, authorization and encryption of data and metadata
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| User Security Manager | Library to add, update, delete users and groups in the security system | not started | medium/4
|
| Resource Security Manager | Library to register add, update, delete resources in the security system | not started | medium/4
|
| Policy Manager | Library to manage and retrieve user and group access rights to resources | initiated | medium/4
|
| Metadata Crypto | Library for the encryption/decryption of XML | initiated | medium/3
|
| Data Crypto | Library for the encryption/decryption of files | not started | medium/2
|
General UtilitiesCollection of various system utilities
| |||
| Name | Description | Status | Other |
|---|---|---|---|
| XMLZip | Library for the compression of XML documents | not started | medium/4
|
| DataZip | Library for the compression of data files | not started | medium/2
|
© 2007-2008, DDI Alliance. Contact/Feedback tools@ddialliance.org
Hosted by the Open Data Foundation