About IDI Search

IDI Search is a web app that allows researchers to search for variables that are available in the IDI and, in some cases, metadata about these variables. The app uses data from IDI variables and Data Dictionaries shared with us by Stats NZ. The data are stored in a database which can then be searched using the web app.

IDI Search has been developed by Te Rourou Tātaritanga, a research group funded by an MBIE Endevaour Grant (ref 62506 ENDRP).

IDI: Integrated Data Infrastructure

The IDI is New Zealand's Integrated Data Infrastructure, provided by Stats NZ. It houses data from government and non-government sources. More information can be obtained from https://www.stats.govt.nz/integrated-data/integrated-data-infrastructure/.

The Update Process

Updates to IDI Search follow the triannual cycle of the IDI refresh with collaboration between Te Rourou Tātaritanga and Stats NZ.

Before each new IDI refresh goes live, Stats NZ extracts the variable listings from the latest refresh and provides this information to the app developers. This allows researchers to build an accurate picture of what data is currently available in the IDI.

Stats NZ also provides the app developers with the latest IDI data dictionaries from the Data Lab User Wiki that have been updated in response to changes and additions to the latest refresh data and researcher queries. This information is extracted and added to the app.

Data Quality

Stats NZ supplies the data included in this app. See the ‘Help’ page for details about the data structure and how to navigate the app.

Currently, the quality of the metadata included in the supplied Data Dictionaries is highly inconsistent between different collections and data supplier agencies. We have tried, where possible, to tidy the information and match as many variables and datasets to collections and agencies as possible.

Stats NZ is proactively working with data supplier agencies to improve metadata quality. However, there is still a long way to go before all of the metadata in the Data Lab is tidied up and formatted consistently so that we can easily import them into our database to display in the app.

If you notice errors, inconsistencies, duplicates, spelling mistakes, or other problems with the information displayed in the app, please contact access2microdata@stats.govt.nz. Your feedback on metadata will facilitate Stats NZ’s continuous efforts to improve metadata quality.

The Future

We want IDI users and those interested in finding out about the IDI to be able to readily access accurate and reliable information about the data contained within the IDI through a user-friendly interface.

The IDI Search app uses a data model that allows for automated updates of data dictionaries, codes, and classifications. In collaboration with Stats NZ, this will ensure the quality of IDI metadata is improved over time.

We encourage researchers to utilise this app and share it with other Data Lab researchers: experienced, new, or potential future researchers! Through your use, we can gain insights that will help further improve the app.

Please contact:


Privacy Statement

We collect analytics data to track page views and search terms. This information is stored internally and used solely for tracking demand and collecting information to help us improve the app.

Disclaimer

Disclaimer for output from Stats NZ surveys or Census data

Access to the data used in this study was provided by Stats NZ under conditions designed to give effect to the security and confidentiality provisions of the Statistics Act 1975. The results presented in this study are the work of the author, not Stats NZ or individual data suppliers.

Disclaimer for output produced from the IDI and/or LBD

These results are not official statistics. They have been created for research purposes from the Integrated Data Infrastructure (IDI) which is carefully managed by Stats NZ. For more information about the IDI please visit https://www.stats.govt.nz/integrated-data/.

Disclaimer for Inland Revenue tax data

The results are based in part on tax data supplied by Inland Revenue to Stats NZ under the Tax Administration Act 1994 for statistical purposes. Any discussion of data limitations or weaknesses is in the context of using the IDI for statistical purposes, and is not related to the data’s ability to support Inland Revenue’s core operational requirements.