DATA MANAGEMENT RESOURCES

CIOOS brings datasets together, making coastal and ocean data findable, accessible, interoperable, and reusable to ocean users.  

Proper management of ocean data benefits all ocean users. Data management with CIOOS makes sure your data can be discovered, increasing the value of your data and reducing the likelihood of duplication. Every single observation in space and time adds to our understanding of the ocean and may be crucial to identifying and describing trends during a time of change.

Data integrated into CIOOS undergo a quality review. This process ensures that the metadata are complete to the level of international standards and catches input errors. CIOOS visualization tools also enable users to inspect data quality before download to make sure the data are appropriate for their needs. 

All data in CIOOS follow the same standards, making it easier to compare apples to apples and oranges to oranges. Therefore, CIOOS allows users to easily access and develop the information they need for decision making. 

At the beginning, middle, and end of a project, planning and taking actions to secure your data legacy can benefit peers, locally or globally, and for generations to come. 

CIOOS is a national partnership for ocean data management expertise. The national structure is staffed out of three regional associations. Our structure ensures that we are in regular communication with our team across the country to solve any issue and improve CIOOS for all. 

In practice, if you email info@cioos.ca we’ll make sure you’re connected with the staff in the region that can best meet your needs. 

If your data are collected on or in the ocean, you’ve come to the right spot and we look forward to helping you. Specifically, CIOOS is looking for data related to, but not limited to, Essential Ocean Variables that are useful to many Canadian ocean users and support efforts to develop a Global Ocean Observing System.

That said, some data types require special handling (e.g., genomic or biological data) and it may be better to submit your data to a repository that is more specific to those data types and then link to those data through CIOOS for enhanced discoverability.

If your data are not observations from on or in the ocean, you may consider checking with an intended journal for their recommendations or consider submitting to Canada’s more general research repositories, Borealis or the Federated Research Data Repository. 

In the data management world, these words have specific meanings that require data storage for a minimum of 10 years. At this time, CIOOS funding cycles cannot guarantee that longevity. That said, CIOOS strives to help the ocean science community gain access to data today, tomorrow, and for as long as funding continues. Your data contribution can help us grow and make CIOOS too valuable to lose.

If you need to have a 10+ year storage solution, CIOOS can still help! Check with an intended journal for their recommendations or consider submitting to Canada’s more general research repositories, Borealis or the Federated Research Data Repository. Once the data are on a server, help the ocean community discover those data by submitting the metadata to CIOOS. Users will then be linked to the appropriate repository.

CIOOS subscribes to the current best practices in data management. Whenever possible we are making data FAIR (Findable, Accessible, Interoperable, Reusable) and strive for TRUST (Transparency, Responsibility, User focused, Sustainability, and Technology). Our application of the FAIR principles in our data management practices also takes into account the sensitivities that data producers may express about their data.

In addition to following the above principles, CIOOS follows best practices in data management that specifically address Indigenous data. These are the CARE (Collective benefit, Authority to control, Responsibility, Ethics) and the First Nations developed OCAP (Ownership, Control, Access, Possession) principles. These principles acknowledge and respect that Indigenous data holders are in control over their data and how it may be stored, used, or shared.

We are working with Indigenous partners to ensure that proper mechanisms reflecting these controls are in place and will continue development as requested to meet partner needs.

To support the movement towards open data, CIOOS offers three licences that data providers may choose from.

• Creative Commons Attribution 4.0 International licence (CC-BY 4.0) – CIOOS recommended. Allows for open sharing and adaptation of the data provided that the original creator is attributed.
• Creative Commons 0 – imposes no restrictions of any kind.
• Open Government Licence – Canada – For datasets made available by Government of Canada departments and agencies, it is very similar to CC-BY as it allows for open sharing and adaptation of the data, provided that the original creator of the data is properly attributed.

Licences are attributed to the dataset when submitting the metadata.

CIOOS is currently researching ways to make proprietary data Findable, but with limited Accessibility. 

Let us know if you’d like to be part of this endeavour at info@cioos.ca.

You may have already submitted your data to another repository… congratulations! You do NOT need to submit everything again to CIOOS. However, you can make your data more findable to end users by submitting the metadata to CIOOS and then linking to the other repository. This is done by adding an entry with our Metadata Entry Tool.

Metadata are the data needed to understand the data itself. This includes everything from the title and abstract of the data, to the time the sample was collected, to the names of the variables observed. Without sufficient and consistent metadata, it becomes challenging for users to find and compare similar observations.

CIOOS has adopted internationally recognized standards for metadata (ISO 19115-1:2014,19115-2:2019, and 19115-3:2016) that allow CIOOS data to be used and reused in conjunction with other ocean observing systems around the world.

To ensure that CIOOS data are consistent, metadata submissions go through a web-based Metadata Entry Tool. This tool guides data providers through both required and optional fields and facilitates language translation.  

CIOOS is a fully bilingual platform. To make discovery possible all data submitted to CIOOS must have complete metadata, in both national languages. We have developed a Metadata Entry Tool that supports you, the data provider, in the translation of relevant metadata fields. We encourage data providers to verify the computer generated translations with experts in the field who have experience in the other language.

The short answer is maybe. CIOOS is not a data archive and does not have the resources to house all datasets on its servers. Most data in CIOOS are stored elsewhere and made discoverable through CIOOS. This has the advantage of reducing the number of data copies in existence. If the data provider updates a dataset in their own system, these changes are automatically reflected to all CIOOS users.

Please contact us at info@cioos.ca if you would like to discuss the needs for your specific dataset(s). 

Data that are stored on CIOOS servers should be formatted in open source formats where possible, such as: 

• CSV
• tabular text files
• JSON
• NetCDF files

A full list of input file types is available here. 

Data in Excel spreadsheets will require format transformation. 

Real-time data requires an API or a public URL for regular access and all data transformations must be fully automated.

Raw data access requires a public URL of the dataset in its original form from the data provider. Ideally the data provider can provide a public URL or API access.

Making metadata interoperable means that datasets may be found by other data portals beyond CIOOS. CIOOS metadata complies with international standards so CIOOS metadata may be harvested by other data portals, such as the Canadian Federated Research Data Repository (FRDR), and the global Ocean Biodiversity Information System (OBIS).  

Making data interoperable means that the data are structured so they can be combined together with other datasets.

Here’s an example of the importance of interoperability: different scientists all working with sea water temperature might record it as: ‘Ocean temp’, ‘Temperature’, ‘Water temperature’, ‘H20 temp’, etc. and the units could be recorded using ‘C’, ‘degrees C’, Celsius, Fahrenheit’, etc. To combine data from multiple sources in this example would require: 

• aggregating variables with many different names for temperature, 
• checking whether the units make sense and can be converted,
• and then converting them. 

To simplify the workflow for end users, CIOOS promotes interoperability standards for all ocean data. Connect with a Data Specialist to discuss what this means for your dataset.

CIOOS aims to use domain-specific standards for every data type. 

For physical observations, CIOOS uses standardized Climate and Forecast (CF) names and canonical units e.g., for “sea_water_temperature” the canonical units are in Kelvin.

In the event that no CF standard name exists for a variable, create a custom variable name of your choice using “variable_names_with_underscores” format. You may consider requesting a new CF variable through their discussion board.

For standard units, any unit (e.g. Celsius) that is convertible to the canonical unit may be used. Units specified must apply to all values in that column. 

Climate and Forecast uses UDUNITS software to provide units and definitions, and standard conversions between unit types. See the current list of requirements for Climate and Forecast standard variables names and use compliance checker tools that exist for ERDDAP™/NetCDF.

For biodiversity data, CIOOS uses the Darwin Core standard. This standard is also used by other data disseminators like the Ocean Biodiversity Information System (OBIS) and the Global Biodiversity Information Facility (GBIF). Primarily based on taxa, observations, samples, and specimens, this standardization simplifies the process of publishing biodiversity datasets and also makes it easier for end users to compare the interoperable data.

For emerging variables of interest, please contact our data specialists to discuss proper naming and formatting of your data.

This may depend on your particular data type. Please check with a CIOOS data specialist if you have any questions. 

That said, for integration of physical observations into ERDDAP™, set up your data table with a single header row titled with Climate and Forecast naming conventions. Each unique variable should have its own column. Columns of overarching application (e.g. platform, scientist, date) should be located to the left of columns containing specific observations (e.g., “sea_water_temperature (degC)”).

Rows 2 and greater should be values from specific observations. (e.g., a temperature of 13.97 °C at a pressure of 10 decibars)

A simple example data table in spreadsheet form (.xlsx) may be found here. Examples of more complex metadata in the format (.xml) ready to be integrated to ERDDAP™ may be found here.

If you are not familiar with submitting biodiversity data read the Darwin Core standards carefully, or reach out to us as standards may differ within the discipline (e.g. occurrence vs tracking data). 

For emerging data our data specialists are happy to work closely with you to ensure that your data are properly formatted for easy integration.

Use ISO 8601 dates:

• YYYY-MM-DD
• 2022-03-09

Use ISO 8601 times:

• 2022-03-09T12:00:00-00:00 or 2022-03-09T12:00:00Z

Always record the time zone that was used. Ideally use UTC time to make time comparisons easier. 

Use decimal degrees for latitude and longitude. 

For example :

CIOOS does not have the resources to make all of Canada’s ocean data interoperable on behalf of the community. That said, our data specialists are more than happy to work with you and your team throughout the data transformation process to field questions you may have. Our experience with large volume data transformation is freely shared with new data contributors and we may have developed tools that can speed up the process. 

Contact us at info@cioos.ca so we can help.

The answer depends on many factors, including how many datasets there are, the current completeness of the metadata, and the capacity of the data provider to prepare the data for submission. CIOOS staff are here to streamline this process, but ultimately the time required is up to you. Contact us at info@cioos.ca to get an estimate that might reflect your situation.

Organizations with properly formatted data ready for ingestion into CIOOS are requested to contact a CIOOS data specialist at info@cioos.ca to facilitate ingestion.

Generally speaking, CIOOS is not a data archive and does not have the resources to house all datasets on its servers. Most data in CIOOS are stored elsewhere and made discoverable through CIOOS. Please contact us if you would like to discuss the needs for your specific dataset(s). 

Data providers are given the opportunity to review submissions of metadata and data on CIOOS development servers before they are made public. Changes may be submitted and data will be made public when authorization has been given.

We currently support metadata identifiers from researchers (Open Researcher and Contributor ID, ORCID) and institutions (Research Organization Registry, ROR). Direct Object Identifiers (DOIs) are supported by the St Lawrence Global Observatory (SLGO) and CIOOS Atlantic.