Tilde, an interdisciplinary data access solution serving GeoNet's volcano and other hazard data

Tilde, an interdisciplinary data access solution serving GeoNet's volcano and other hazard data | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Tilde, an interdisciplinary data access solution serving GeoNet's volcano and other hazard data Elisabetta D’Anastasio, Joshua Groom, Steven Sherburn, Jeremy Houltham, and 6 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4190192/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 28 Nov, 2024 Read the published version in Bulletin of Volcanology → Version 1 posted 5 You are reading this latest preprint version Abstract Tilde is a new in-house developed solution that the GNS Science’s GeoNet programme has recently developed to provide storage and access to low sample rate datasets used to monitor tsunami, landslides and volcanoes in Aotearoa New Zealand. It includes datasets covering sample rates of minutes or longer. Time series data are stored and disseminated in JSON and CSV formats, and users can access these through an Application Programming Interface (API) and a Graphical User Interface (GUI). Tilde’s GUI was created to allow technical and non-technical users easy access to the available data. The introduction of Tilde system as one of the GeoNet programme delivery channels has represented a big step forward for GeoNet’s volcano data holdings by providing a single point to access all low to medium sample rate volcano specific monitoring data. We designed the system, developed a domain model, an API, a graphical data discovery interface and associated data tutorials. This work leverages the Open by Default data policy for data generated through the GeoNet programme. This paper is intended to highlight how we made many of the key decisions that shaped Tilde system, how they were impacted by our multi-hazard monitoring requirements and how they have improved access to volcano monitoring data. We conclude with some open questions about the need of developing common standards to share analysis-ready time series data within different disciplines in volcanology and geophysics. volcano monitoring data time series data interoperability data ingestion and dissemination application programmatic interface graphical user interface data storage Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Introduction GeoNet is a programme within GNS Science Te Pῡ Ao (GNS hereafter) that monitors Aotearoa New Zealand volcanoes, earthquakes, landslides and tsunami using a diverse range of instrumentation and methodologies. Volcano monitoring data from seismic, geodetic, and multi-parameter environmental sensors are continuously recorded, collected, processed, and made available to researchers and the public. GeoNet also hosts and disseminate observations that are manually collected by GNS scientists on active New Zealand volcanoes. Data are collected at different time intervals (or sampling rates), from fraction of seconds to days/months. Details of this wealth of data and how they are used by the GNS Volcano Monitoring Group (VMG hereafter) are provided in Hanson et al., this volume. Open data by default is at the core of GeoNet data principles and all systems and tools are built by the programme so that data are immediately available after collection at the data center. Some of the terminology used here to describe the GeoNet data pipeline, is described in the “GeoNet Aotearoa New Zealand Glossary of Data-related terms” (Abbott et al., 2021). GeoNet volcano, earthquake, landslide and tsunami monitoring data comes from different scientific disciplines. These data are archived and disseminated to end users with access mechanisms and formats that are domain-specific, as this allows a full representation of the data and associated metadata. Experts in each of these fields can leverage existing tools and resources to retrieve, process and interpret the data, and will have the flexibility to tailor the processing to the specific needs of their analysis. This works well for specialised users who focus and use data from a specific domain in which they are experts but can prevent multidisciplinary studies or require collaboration across experts from multiple domains to obtain a full picture of observations from different sensors installed at a volcano. The data recording, collection, processing and dissemination infrastructure that GeoNet operates provides data from several disciplines to scientists and monitoring groups. When possible, data users would benefit from an analysis ready, easy to use and common data and metadata format disseminated through a common platform. Disciplines used in volcanology traditionally suffer from a lack of common and standardized data formats and access mechanisms for volcano data. Monitoring agencies and researchers spend a significant amount of time in searching, downloading, validating, processing and cleansing the data before they can start their analysis and interpretation. Lowenstern et al. ( 2022 ), in their guidelines and recommendations for volcano observatory operations during crises, point out that, during a crisis, “it is fundamental to process and visualize scientific data in an integrated manner, over the same timeline and with common axes to integrate data collected at different frequencies” and that “ideally these systems incorporate multidisciplinary data that can be used by a broad range of observatory scientists who participate in analysis and evaluation of insightful correlations”. When such integrated systems are available, often they are not accessible by scientists that are outside the monitoring agency. Initiatives such as WOVOdat ( https://www.wovodat.org/ , Venezky and Newhall, 2007 ) from the World Organization of Volcano Observatories or the Volcano Observation Thematic Core Service of the European Plate Observing System (EPOS TCS VO, Puglisi et al., 2022 ), are trying to encompass this issue, and provide free and open access to a series of data submitted by volcano observatories around the world. But, to the best of our knowledge, individual observatories and scientists wanting to monitor and research volcanoes in their country generally use a diverse range of tools, databases and access mechanisms to collate and analyse all data they have available. To facilitate interdisciplinary and open data access for Aotearoa New Zealand volcanoes, and following FAIR principles (Wilkinson et al., 2016 ), we developed a common platform called Tilde (Time Lapse Data Entries) to ingest, store, disseminate and visualize volcano data collected by the continuous GeoNet sensor network and the VMG. We will introduce and describe the Tilde storage system and associated ingestion and access applications, using examples of volcano monitoring data. It is important to note that even though Tilde is the primary access mechanism for GeoNet volcano data, it also provides data for tsunami and landslide monitoring. We want to expose Tilde to volcanologists who might want access GeoNet data, and to those who might develop or need to understand similar systems. Solution Design and Requirements Volcano monitoring data have been collected by GNS Science and archived and disseminated by the GeoNet programme since its inception in 2001 (Hanson et al., this volume). Those are continuously recorded and automatically collected or manually sampled and ingested in the GeoNet data ecosystem at discrete time intervals. There are differences between how automatically and manually collected data are dealt with. Except for seismic and GNSS data, all GeoNet volcano monitoring data typically have a recording rate of once per minute or longer and are defined as “low-rate data”. Raw and processed data from continuous seismic, acoustic and geodetic stations are used for monitoring volcano and other hazards and are archived and disseminated following community standards. Data that are specific to volcano monitoring have not always been treated in a consistent way and often lack from a widely shared community standard. GeoNet recently developed systems to automatically collect ScanDOAS (Mazot et al., 2017 ; GNS Science, 2022) and volcano environmental monitoring data for fumaroles, springs and lakes (GeoNet, 2023 ), and these are treated like other automatically collected data. For discrete datasets, the VMG manually sample and analyse volcano features, generating a wide range of observations that are typically handled on spreadsheets and curated and archived on GNS premises. GeoNet will then collate the analysis results and distribute those to all data users. The multiparametric observations collected at Aotearoa New Zealand active volcanoes are a fundamental tools for the VMG, and it is evident that an application that allow an integrated view of multidisciplinary data was of critical importance for the GeoNet programme and the VMG. Following the FAIR principles for scientific data (Wilkinson et al., 2016 ) and leveraging the open access and CC-by license attributed to all GeoNet data (Hanson et al., this volume), we designed Tilde to improve findability, accessibility and interoperability of volcano and other low-rate time series data. We list below some of the design principles, but it shall be noted that the implementation of some of the Tilde components has been largely influenced by the earliest timeseries datasets that went into the system. As some of the low-rate time series data are costly to reproduce and, in some cases, (especially for manually collected volcano) the raw data are perishable, time series data needed to be archived to ensure data longevity and minimize the risk of data loss. The chosen data distribution format needed to be universally comprehensible, consistent and suitable across diverse domains. While Tilde was not designed as a metadata delivery application, serving a simplified description for each time series was developed for users who may not possess expertise in all relevant disciplines and to provide a common metadata view to programmatic access. During the development phase of Tilde, findability and accessibility of GeoNet time series data were at the forefront of our design philosophy. The ability to provide a system to integrate and disseminate data from different disciplines was important to support and facilitate volcano monitoring. A simple and intuitive data discovery solution, with user-friendly browsing features and a common perspective across different domains, was strongly required by our data users. This was one of the main drivers of designing a system that could support a user friendly and flexible Graphical User Interface (GUI) and facilitate data discovery and exploration. In fact, the wealth of GeoNet volcano monitoring data that has been historically not easy to explore and access (Hanson et al., this volume). We followed User Centred Design principles (UCD), and designed the Tilde GUI in 3 key phases: 1) a research discovery phase, during which we gathered requirements from key user groups to understand what the priority features for the GUI were; 2) a prototype design phase, during which we iterated on different design solutions to find the optimal design that worked best for the majority of end users and did not create undue load on the Tilde system; 3) a testing phase, during which we conducted one to one user testing with selected users to ensure the GUI performed as it should and find any bugs that may be present. The GUI prototype design phase ensured we had all the elements needed to meet the requirements of our end users. Design requirements also included considerations about the IT infrastructure that the Tilde system was going to rely on. Tilde backend code and infrastructure must be maintained and enhanced within GeoNet standard practices. Because GeoNet handles multi-hazard data, with a variety of types and processing systems, our preference to design a common system for time series data was to develop an in-house solution, using program languages and IT infrastructures common to other GeoNet systems. Sustainability considerations also included the ongoing cost of maintaining required storage or databases in GeoNet’s preferred cloud storage environment. Tilde was developed by a team composed of software developers, IT engineers, subject matter experts, system architects and user experience designers. An extended consultation with end users, including the VMG and technical experts gathered functional and non-functional requirements. Tilde solution Design of logical model The design of Tilde started with developing a domain model that could be used to describe all concepts pertinent to the GeoNet data use cases. While doing that, we explored open standards and vocabularies used to describe time series data. The Open Geospatial Consortium (OGC) is an international consortium that develops and publishes standards for geospatial and location-based services. OGC developed the TimeSeriesML (https://www.ogc.org/standard/tsml/) and SensorML (https://www.ogc.org/standard/sensorml/) standards. We explored both, but unfortunately, we were not able to fully model our metadata requirements using TimeseriesML (too generic) or SensorML (too sensor oriented), to fully comply to all our requirements. The cost and skillsets required to develop an OGC compliant systems were not available to us within a workable timeframe. The WOVOdat vocabulary and schema (Venezky and Newhall, 2007) was, on the other hand, too specific to volcano data for our multi-hazard requirements of Tilde and we were not able to utilize the WOVOdat vocabulary and schema either to cover all the Tilde use cases. Furthermore, the VMG, our largest volcano data user, does not use WOVOdat vocabulary and schema so trying to incorporate parts of that into how Tilde was used for volcano data did not offer any advantages. We opted for an in-house developed logical model and a vocabulary. The model has been designed to rationalize, map out and name all common concepts we needed to describe GeoNet time series data and how their entities are related. There are benefits and drawbacks to this decision, which we will touch on later. The Tilde logical model is based on the concepts of domains, series, “data objects” and their attributes (Fig. 1). A “ series ” is composed by all observations recorded by a sensor through time and processed to obtain measurements of a specific parameter. Each data object is used to describe individual observations at a specific point in time, and includes a timestamp, an observation value, an observation error and a quality control flag. While the quality control flag is available, we have not yet begun to use it, though works is underway to do this. Different series are grouped under the same domain . A domain is directly related to the type of discipline and sensor used to record the data object (for example environmental sensor). A series is uniquely identified by a number of attributes that include: a) the station identifier, where the sensor is installed or manual observations is made; b) a sensor code identifier, used to distinguish between different sensors at the same location, or to differentiate changes in the same type of sensor through times, when those changes are significant and might impact the time series; c) a series name , used to describe what is measured by the data object (for example water temperature); d) a method , used to describe the methodology used to collect the measurements or their time resolution, for example 15 minutes sampling rate, or flyspec collection system; e) an aspect , used to differentiate between features of the same time series, if necessary to further disambiguate the time series. Additional attributes are provided to describe the station metadata, and properties of all data objects in a series. Figure 1 provide a simplified snapshot of the Tilde logical model and some additional properties. A detailed and complete description of all properties is provided in the API Tilde documentation (GeoNet, 2022). Figures 2 and 3 provides some real-world examples of how the logical model has been applied to two types of volcano monitoring time series. We provide one example for continuous data and one example for manually collected data to show what are the fundamental entities used to identify a time series, and how they can be applied for different scenarios. Figure 2 illustrates entities of a timeseries that is generated by a datalogger measuring the temperature of a fumarole. The datalogger is part of the “envirosensor” domain (GeoNet, 2023; GNS Science, 2018). The datalogger is installed at a station , and the sensors in the nearby fumarole. The datalogger will record a snapshot of the temperature every 10 minutes and a maximum and minimum value over a 10-minutes interval. Different sensors are identified with the sensor code attribute. Each sensor will thus generate a uniquely identifiable time series, that can be grouped under the same station and name . The same station often has other types of sensors measuring different environmental observations. Figure 3 illustrates a similar use case, but for the manually collected data domain (or “manualcollect”; GNS Science, 1954). In this case, the sensor code attribute will be used to distinguish “spring-temperature” observations taken at slightly different points in the same area ( station ), for example spring source and spring outflow, with a thermocouple sensor. For “manualcollect” timeseries, the aspect attribute is rarely used, but can differentiate observations derived from multiple samples at one time from the same feature, for example chloride concentration from spring water where all samples and analyses provide slightly different but equally valid values. A complete description of the tilde domain model and all its entities is provided through the Tilde API documentation (GeoNet, 2022). System Architecture A detailed description of the tilde system architecture is out of scope of this paper, but a simplified overview is provided in Fig. 4. The majority of GeoNet data processing systems are running on Amazon Web Services (AWS) cloud infrastructure. In house developed software, configurations and databases are maintained on GitHub and systems integration and deployments are orchestrated by several processes and coded infrastructure. GeoNet data storage is relying on AWS Simple Storage Service (S3), and applications will generally utilize scheduled tasks or messaging systems and queues to run processes. A metadata database common to all GeoNet sensors networks (GNS Science, 2019) describes instruments and station metadata that all applications rely on. GeoNet data collection and processing systems separate from Tilde handle the continuous data feeds (at sampling rates that depends on the data domain) and generates time series observations. Similarly, VMG curate and process manually collected data separate from Tilde. Tilde time series metadata attributes are automatically extracted from the common metadata database (GNS Science, 2019), that is also used as primary source for systems that perform data processing. Once the continuous (or sampled) data processing is completed, each time series data point is formatted with a JSON structure compatible with Tilde and loaded in a S3 “spool” bucket. A Tilde ingest service then consumes the data point and associates it with the correct time series metadata by using the time series unique identifiers attributes and stores it on S3. Tilde uses the files stored on S3 as a form of database. This proves to be both cost effective and scalable (supports many concurrent users) however as time series grow many accumulated individual data points must be routinely repackaged into monthly and yearly files (a process referred to as “bundling”) to keep performance acceptable. The current design limits Tilde to time series with 15 second or greater time resolution, further evolutions of Tilde will improve on this, if there is a sensible use case. We anticipate that there will be future evolution of the Tilde system, and code, storage and frontend solutions. As such, Tilde has been designed to support versioning and upgrades whilst allowing continuity and migration of downstream processes. At the time of writing, we are currently at version 3 of Tilde. Storage Tilde time series bundles are stored in Comma Separated Values (CSV) format, a widely used, open and flexible format. To enable machine learning applications and following recommendations for big datasets on cloud infrastructures, we adopted a self-documenting naming convention to store the timeseries bundles. Time series objects in the S3 Tilde archive follow a hive style partitioning in their name, where time series attributes are explicitly presented in the archived S3 object. Tilde bundled Time series data are made available through the AWS Open Data Program (GeoNet, 2022A) and can be accessed via the s3://geonet-open-data/time-series/tilde/ S3 bucket. Timeseries are organized by version, domain, station, time series name, sensor code, time series parameters (method and aspect) and time. A time series bundle naming convention is as follows (the name is splitted below as bullet point list for readability): time-series/tilde/[v]/domain= [domainkey] / station= [stationkey] /name= [namekey] /sensorcode= [sensorcodekey] / method= [methodkey] /aspect= [aspectkey] / start= [YYYY-MM-DD] / [domainkey].[stationkey].[namekey].[sensorcodekey].[methodkey].[aspectkey].[unit].[errorunit].[YYYY-MM-DD] T00:00:00Z.csv.gz In this naming style, “ v ” is the Tilde system version and domain, station , name , sensorcode , method and aspect have been already discussed. “ start ” is the starting point of the time window of the time series bundled. We chose to slice the data monthly as a compromise among the range of sampling rates of the timeseries types we currently handle. The remaining part of the S3 object name will repeat these information and add time series unit attributes. Hive-style naming should allow programs and applications to easily select and slice time series observations of interest. The S3 geonet-open-data access mechanism (GeoNet, 2022A) is the recommended one for those who want large volumes of data and is the most effective to use for bulk downloads. Application Programmatic Interface The Tilde frontend interface is an Application Program Interface (API). By default, the Tilde API will return information or data in JavaScript Object Notation (JSON) format. JSON is a standard text-based format that represents structured data, is human readable but is oriented to programmatic use. The Tilde API supports three endpoints that are used to query timeseries data, statistics and metadata. The data endpoint (GeoNet, 2022B) is the mechanism to query time series data. The Tilde API allow flexibility in how timeseries data are queried, with options to slice by time interval, observation name, method and so on. Some of the attributes ( sensorCode , method and aspect ) can also be wildcarded by using the “-” symbol, allowing requestors to download all available time series for a given station and series name without having to know in advance all attributes of the data they’re interested in. The stats endpoint (GeoNet, 2022C) is the mechanism to query time series observations statistics such as first and last observation, number of records, maximum, minimum, mean values and some other simple statistical calculations. This endpoint is useful for a quick inspection of the timeseries of interest and can be used to assess or do simple and “pre-canned” analysis of the observations. The dataSummary endpoint (GeoNet, 2022D) is a flexible mechanism to query and explore information about what timeseries are available. The endpoint can be used to query what domains are available, stations available for a certain domain, sensorCodes available for a certain station and so on. An overview of the time interval covered, number of series, and associated stations, and more specific time interval for each time series name is also provided. Each endpoint will return standard HTTP response status code to provide information on successful requests (HTTP status code 200), malformed requests (HTTP status code 400) and so on. Even if the API can serve bulk downloads and large data requests, we are encouraging those interested in large volumes of data to use the S3 stored time series bundles available on the GeoNet AWS Open data bucket (GeoNet, 2022). Graphical User Interface The Tilde graphical interface is built as a Data Discovery GUI (GeoNet, 2022E), following key requirements discovered during the design research phase. The Data Discovery GUI allows users to easily find what datasets are available in Tilde, what time periods they are available for, view the time series on an interactive graph and allow for data to be downloaded as a CSV file or in JSON format. The elements on the Data Discovery GUI consist of 1) a data table with relevant domain and timeseries information; 2) a map with station icons and 3) a data query build element with dropdown selections to allow the user to select the time frame of interest and a data aggregation function (Fig. 5). Once the data query elements are selected, the user can then view the time series on a graph, download the data in JSON or CSV format, and view how the same request needs to be programmed to use the Tilde API in their own application. The design phase clearly indicated the need of data users interested in using the API to have a mechanism to quickly understand how an intuitive graphical selection shall be handled programmatically. The Tilde Data Discovery GUI testing phase was crucial to uncover bugs in the design that made it difficult for user to perform tasks and request the data they were looking for. The iterative approach used to develop the Tilde GUI allows as to adjust the GUI design accordingly. We implemented some limitations on the amount of data that can be viewed using the GUI to optimize its performance. When a GUI plotting query is too large, a response message is provided to the user that recommends using the API or the AWS Open Data access mechanisms. The Tilde GUI has been designed in a modular way to allow for the creation of peril specific web-pages on the GeoNet website. This allows our developers to re-use the same elements created for the Data Discovery GUI to create dedicated data discovery pages for specific domains. Website pages dedicated to volcano monitoring will be developed in the future. Tutorials Increased usage of the low and medium sample rate data is at the heart of Tilde development. As such, alongside a detailed API documentation, we have developed a series of tutorials that showcase how the API can be used in python and bash codes. Those are publicly available via the GeoNet GitHub data-tutorials repository (GeoNet, 2020) under a dedicated section for Tilde. We would like to encourage anyone using these to contribute to the data tutorials repository and provide input if they wish. In this way, GeoNet is encouraging data users to also support each other. GeoNet has recently started producing Data Blogs (https://www.geonet.org.nz/news; filter for Data Blogs) to illustrate the diversity of GeoNet data, showcase some interesting use cases and support with examples the understanding of how to interact with them. To date, several data blogs have focused on data that can be accessed via Tilde, and many of them with a volcano data focus. These include analysing some of the spring-based water height and temperature monitoring (GeoNet, 2023A), accessing manually collected data (GeoNet, 2024) and using a lake level monitoring site on Lake Taupo, installed to capture tsunami generated within the lake in Taupo caldera (GeoNet, 2023B). More data blogs will continue to be produced building on Tilde and its data delivery opportunities, among other topics. End-users were at the heart of the Tilde design and implementation. Features like the generation of an API request for a given GUI data search were added based on user feedback and significantly lower the barrier for programmatic data access for some users. And combining data access via a GUI, API and through GeoNet AWS Open Data provide for a wide swath of use cases, ranging from very specific data analyses to large machine-learning initiatives. Conclusion Tilde data harmonization, storage and dissemination solution has allowed the GNS’s GeoNet programme to provide a common platform for users to access and visualize time series data. Efforts towards the development of the Tilde GUI massively improves technical and non-technical user and browsing experiences and provides easy access to the available data. Flexibility in the API endpoints allow users to interrogate all parameters available for any data stream and tweak the programmatic download when querying data for further data analysis. This system has improved the Findability and Accessibility of GeoNet volcano monitoring data (continuously and manually collected) and FAIR principles (Wilkinson et al., 2016 ) have been used as a tool to guide our IT system design. The flexibility of the API and the inclusion of a versioning mechanism will improve the experience of users downloading the data programmatically, alongside the documentation and tutorials to illustrate how to interact with it. The hive-style name partitioning and the availability of Tilde time series on a cloud storage system (AWS S3), will hopefully act as enablers of machine learning applications in the future. For volcano monitoring data, Tilde will provide all time-series monitoring data. As of March 2024, it provides virtually all manually collected data and envirosensor data, and work to add sulphur dioxide fluxes from scanDOAS (Hanson et al., this volume) is underway. Ground deformation data from GNSS, for volcano and tectonic monitoring will be added within a year. Due to lack of community standards in the geophysics and volcanology domains, and the effort required to implement and promote an OGC compliant standard, we ended up developing an in-house domain schema, not interoperable outside of GeoNet. This is not ideal, as analysis ready time series data could be potentially used by several disciplines if a common and interoperable standard was available and used. With the constant increase of open data, sensor devices and ability to easy share data globally, the geophysics community and its various disciplines should put efforts towards combining knowledge across various domains and build a common standard to share observations from a diverse range of geophysics disciplines. Initiatives such as EPOS ( https://www.epos-eu.org/ ), the recently formed EarthScope consortium ( https://www.earthscope.org/ ), and the increasing focus on data interoperability in the geophysics community will hopefully bridge that gap in coming years. Declarations Acknowledgments GeoNet is sponsored by the New Zealand Government through its agencies: Toka Tū Ake EQC, GNS Science Te Pū Ao, Toitū Te Whenua Land Information New Zealand (LINZ), the National Emergency Management Agency (NEMA) and the Ministry of Business, Innovation and Employment (MBIE). The authors wish to thank all colleagues that provided input and tested the system during its development, and in particular the GNS Science Volcano Monitoring Group and Tsunami teams. Funding This research was supported by the GeoNet programme through its funding agencies Toka Tū Ake EQC, GNS Science Te Pū Ao, Toitū Te Whenua Land Information New Zealand (LINZ), the New Zealand National Emergency Management Agency (NEMA) and the New Zealand Ministry of Business, Innovation and Employment (MBIE). Conflicts of interest/Competing interests All authors declare that they have no conflicts of interest to disclose. Availability of data and material Tilde Application Programmatic Interface is documented and available through the GeoNet website (https://www.geonet.org.nz/). Tilde data holdings are openly available under a CC-by license through the Tilde platform (https://tilde.geonet.org.nz/). Code availability not applicable References Abbott E, GeoNet (2021) team GeoNet Aotearoa New Zealand Glossary of Data-related terms [Dataset], GNS Science, https://doi.org/10.21420/XQS0-0Z48 Lowenstern JB, Wallace K, Barsotti S et al (2022) Guidelines for volcano-observatory operations during crises: recommendations from the 2019 volcano observatory best practices meeting. J Appl Volcanol 11:3. https://doi.org/10.1186/s13617-021-00112-9 GeoNet (2020) GeoNet Data Tutorials. GeoNet repository on GitHub. https://github.com/GeoNet/data-tutorials/ . Accessed 30 March 2024 GeoNet (2022) AWS Open Data Access. GeoNet website https://www.geonet.org.nz/data/access/aws . Accessed 30 March 2024 GeoNet (2022A) Tilde Application Programming Interface. GeoNet website. https://tilde.geonet.org.nz/ . Accessed 30 March 2024 GeoNet (2022B) Tilde Application Programming Interface Observation Data endpoint. GeoNet website. https://tilde.geonet.org.nz/v3/api-docs/endpoint/data . Accessed 30 March 2024 GeoNet (2022C) Tilde Application Programming Interface Observation Statistics endpoint. GeoNet website. https://tilde.geonet.org.nz/v3/api-docs/endpoint/stats . Accessed 30 March 2024 GeoNet (2022D) Tilde Application Programming Interface Data Summary endpoint. GeoNet website. https://tilde.geonet.org.nz/v3/api-docs/endpoint/dataSummary . Accessed 30 March 2024 GeoNet (2022E) Tilde Time Series Data Discovery. GeoNet website. https://tilde.geonet.org.nz/ui/data-exploration . Accessed 30 March 2024 GeoNet (2023) GeoNet Environmental Data. GeoNet website https://www.geonet.org.nz/data/types/environmental . Accessed 30 March 2024 GeoNet (2023A) Using Our Own Data – Hot Spring Water Level and Temperature. GeoNet website https://www.geonet.org.nz/news/2YpqCnWiYCMUEkaPi9ZJj3 . Accessed 30 March 2024 GeoNet (2023B) New Seismic, GNSS and Tsunami Monitoring Sites at Lake Taupo – How Delta Drives GeoNet’s Data Processing and Discovery. GeoNet website https://www.geonet.org.nz/news/2bY6I9tix34IANrWcI9s18 . Accessed 30 March 2024 GeoNet (2024) Access to Manually Collected Volcano Data is Moving from the FITS API to the Tilde API. GeoNet website https://www.geonet.org.nz/news/26beUFBJGh5t03JtoWtRft . Accessed 30 March 2024 GNS Science (1954) GeoNet Aotearoa New Zealand Manually Collected Volcano Data [Dataset]. GNS Science. https://doi.org/10.21420/PSP7-KW60 GNS Science (2018) GeoNet Aotearoa New Zealand Automatically Collected Volcano Data [Dataset]. GNS Science. https://doi.org/10.21420/EN0F-XY29 GNS Science (2019) GeoNet Aotearoa New Zealand Stations Metadata Repository [Dataset]. GNS Science, GeoNet. https://doi.org/10.21420/0VY2-C144 Mazot A, Christenson BW, Scott BJ (2017) Mini-DOAS data processing for SO2 emission measurements and monitoring volcanic degassing on White Island (Whakaari), New Zealand. p. 668 in: IAVCEI 2017 Scientific Assembly abstracts: fostering integrative studies of volcanoes, August 14–18, Portland, Oregon, U.S.A. IAVCEI Puglisi G, Reitano D, Spampinato L, Vogfjörd KS, Barsotti S et al (2022) The integrated multidisciplinary European volcano infrastructure: from the conception to the implementation. Ann Geophys, 65 (3), pp.DM320. ⟨10.4401/ag-8794⟩. ⟨hal-03958096⟩ Venezky DY, Newhall CG (2007) WOVOdat design document; the schema, table descriptions, and create table statements for the database of worldwide volcanic unrest (WOVOdat Version 1.0): U.S. Geological Survey Open File Report 2007 – 1117, 184 p. http://pubs.usgs.gov/of/2007/1117/ Wilkinson M, Dumontier M, Aalbersberg I et al (2016) The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3:160018. https://doi.org/10.1038/sdata.2016.18 Cite Share Download PDF Status: Published Journal Publication published 28 Nov, 2024 Read the published version in Bulletin of Volcanology → Version 1 posted Editorial decision: Minor Revisions 20 Aug, 2024 Reviewers agreed at journal 23 Apr, 2024 Reviewers invited by journal 19 Apr, 2024 Editor invited by journal 18 Apr, 2024 First submitted to journal 29 Mar, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4190192","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":293003676,"identity":"9b197c6f-7371-44fa-89cf-0f969dd97df7","order_by":0,"name":"Elisabetta D’Anastasio","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABAklEQVRIiWNgGAWjYHACAyjN2MDwAco8QIwWCZAWxhkkamFgYOYhxlX8DMwbPxfU1NXx9x9ufGzz6449A/vZhwcYd9Th1CLZwFYsPePYYQmJG4nNxrl9zxIbeNINDjCeYcPtqgM8BtI8bAckGG4wtknn9hxOYJBgYzjA2IbbjfYHeIx/8/yrk5A/f7BN2rLnsD1UiwRuWxh4zKR525glDA4ktkkz/DjM2ADRYoBTi8RhtjJr3r7DkhuBfjHsbTic2MaTxgDUnoBTC3978+bbPN/q+OXOH3/44Mefw/b87MeYP3xswx1iDMzIHMY2BgZwUOG2AwP8IV7pKBgFo2AUjBwAAAFcTmssgWx2AAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0003-3043-8604","institution":"GNS Science Ltd","correspondingAuthor":true,"prefix":"","firstName":"Elisabetta","middleName":"","lastName":"D’Anastasio","suffix":""},{"id":293003677,"identity":"f6364ca0-7f17-419e-9192-c1ec1999a195","order_by":1,"name":"Joshua Groom","email":"","orcid":"","institution":"GNS Science Ltd","correspondingAuthor":false,"prefix":"","firstName":"Joshua","middleName":"","lastName":"Groom","suffix":""},{"id":293003678,"identity":"1f7c96d6-7b14-44c9-8689-760b1e5df422","order_by":2,"name":"Steven Sherburn","email":"","orcid":"","institution":"GNS Science Ltd","correspondingAuthor":false,"prefix":"","firstName":"Steven","middleName":"","lastName":"Sherburn","suffix":""},{"id":293003679,"identity":"44bd8735-bf7a-4af2-9f3b-12c5282c572e","order_by":3,"name":"Jeremy Houltham","email":"","orcid":"","institution":"GNS Science Ltd","correspondingAuthor":false,"prefix":"","firstName":"Jeremy","middleName":"","lastName":"Houltham","suffix":""},{"id":293003680,"identity":"404395cb-e79a-4040-82a3-cf512cc3d7d5","order_by":4,"name":"Mark Chadwick","email":"","orcid":"","institution":"GNS Science Ltd","correspondingAuthor":false,"prefix":"","firstName":"Mark","middleName":"","lastName":"Chadwick","suffix":""},{"id":293003681,"identity":"8b8b4619-63d8-4e04-a2e5-2271642fc94d","order_by":5,"name":"Callum Morris","email":"","orcid":"","institution":"GNS Science Ltd","correspondingAuthor":false,"prefix":"","firstName":"Callum","middleName":"","lastName":"Morris","suffix":""},{"id":293003682,"identity":"2785560e-c39c-44c0-b26a-688f204d3d45","order_by":6,"name":"Baishan Peng","email":"","orcid":"","institution":"GNS Science Ltd","correspondingAuthor":false,"prefix":"","firstName":"Baishan","middleName":"","lastName":"Peng","suffix":""},{"id":293003683,"identity":"d5d8aa0a-768a-42d2-8b14-d185f1271abc","order_by":7,"name":"Howard Wu","email":"","orcid":"","institution":"GNS Science Ltd","correspondingAuthor":false,"prefix":"","firstName":"Howard","middleName":"","lastName":"Wu","suffix":""},{"id":293003684,"identity":"6fc7f6f3-3008-4d9e-917a-b80ea175fef7","order_by":8,"name":"Sue Harvey","email":"","orcid":"","institution":"GNS Science Ltd","correspondingAuthor":false,"prefix":"","firstName":"Sue","middleName":"","lastName":"Harvey","suffix":""},{"id":293003685,"identity":"ea43341b-3392-4481-ae1f-146275461b8c","order_by":9,"name":"Jonathan Bruce Hanson","email":"","orcid":"","institution":"GNS Science Ltd","correspondingAuthor":false,"prefix":"","firstName":"Jonathan","middleName":"Bruce","lastName":"Hanson","suffix":""}],"badges":[],"createdAt":"2024-03-30 02:02:37","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4190192/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4190192/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s00445-024-01786-w","type":"published","date":"2024-11-28T15:57:43+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":55160781,"identity":"13de7b3a-ef23-4b6f-a804-0116d4552172","added_by":"auto","created_at":"2024-04-23 12:58:55","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":16762,"visible":true,"origin":"","legend":"\u003cp\u003eGraphical representation of the Tilde domain model, showing how entities and attributes are related to each other.\u003c/p\u003e","description":"","filename":"OnlineFig1.png","url":"https://assets-eu.researchsquare.com/files/rs-4190192/v1/20bce0d0b396a2a81bd086cc.png"},{"id":55161313,"identity":"97db53d1-4dfa-471e-b6a0-368dea4aeaa4","added_by":"auto","created_at":"2024-04-23 13:06:55","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":64326,"visible":true,"origin":"","legend":"\u003cp\u003eExample of the tilde domain model and entities applied to continuously collected volcano monitoring data for the environmental sensors' domain. The example represents a station where air-temperature and fumarole-temperature are measured by 1 and 3 sensors, respectively.\u003c/p\u003e","description":"","filename":"OnlineFig2.png","url":"https://assets-eu.researchsquare.com/files/rs-4190192/v1/54d0b2dfd2e86b033ba812b9.png"},{"id":55160780,"identity":"2743bd58-9245-449e-a591-7e2458db2686","added_by":"auto","created_at":"2024-04-23 12:58:55","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":44935,"visible":true,"origin":"","legend":"\u003cp\u003eExample of the tilde domain model and entities applied to manually collected volcano monitoring data. The example represents a spring temperature sample taken using a thermocouple.\u003c/p\u003e","description":"","filename":"OnlineFig3.png","url":"https://assets-eu.researchsquare.com/files/rs-4190192/v1/27063408acc0a70c59c17498.png"},{"id":55160784,"identity":"97a5fad5-6c37-4afc-8bab-09c4a04d498a","added_by":"auto","created_at":"2024-04-23 12:58:55","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":84631,"visible":true,"origin":"","legend":"\u003cp\u003eSimplified architecture diagram of the Tilde platform and systems that provides data to it. Tilde components are represented with orange colors, green colors represent systems that are directly interacting with Tilde, and black and white symbols represent systems that are external to Tilde but crucial for its functionalities. Bucket symbols indicate an AWS Simple Storage Service (S3) bucket, and orange rectangular symbols indicate an AWS Simple Queue Service (SQS) and related application.\u003c/p\u003e","description":"","filename":"OnlineFig4.png","url":"https://assets-eu.researchsquare.com/files/rs-4190192/v1/ff54495766585c948141e54c.png"},{"id":55161314,"identity":"06486fae-a1a6-432e-a4e0-070eedd25c6b","added_by":"auto","created_at":"2024-04-23 13:06:55","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":42229,"visible":true,"origin":"","legend":"\u003cp\u003eExample of use of the Tilde Data Discovery Graphical User Interface as captured in March 2024 (GeoNet, 2022E). The example shows the map and data query and plotting selection elements (“Build your Data Query” section in GeoNet, 2022E). In this example, the fumarole-temperature recorded by two sensors installed at the station NA002 (Ngauruhoe Outer Rim Fumarole) are plotted for the time period of March 2024.\u003c/p\u003e","description":"","filename":"OnlineFig5.png","url":"https://assets-eu.researchsquare.com/files/rs-4190192/v1/f5a8079adf37e071a2f04b94.png"},{"id":70382761,"identity":"b0b6db1a-de7e-42c4-b46e-ddad025fd260","added_by":"auto","created_at":"2024-12-02 16:30:31","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":723712,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4190192/v1/1206b989-3eda-400e-a779-d237fa16ddce.pdf"}],"financialInterests":"","formattedTitle":"Tilde, an interdisciplinary data access solution serving GeoNet's volcano and other hazard data","fulltext":[{"header":"Introduction","content":"\u003cp\u003eGeoNet is a programme within GNS Science Te Pῡ Ao (GNS hereafter) that monitors Aotearoa New Zealand volcanoes, earthquakes, landslides and tsunami using a diverse range of instrumentation and methodologies. Volcano monitoring data from seismic, geodetic, and multi-parameter environmental sensors are continuously recorded, collected, processed, and made available to researchers and the public. GeoNet also hosts and disseminate observations that are manually collected by GNS scientists on active New Zealand volcanoes. Data are collected at different time intervals (or sampling rates), from fraction of seconds to days/months. Details of this wealth of data and how they are used by the GNS Volcano Monitoring Group (VMG hereafter) are provided in Hanson et al., this volume.\u003c/p\u003e \u003cp\u003eOpen data by default is at the core of GeoNet data principles and all systems and tools are built by the programme so that data are immediately available after collection at the data center. Some of the terminology used here to describe the GeoNet data pipeline, is described in the \u0026ldquo;GeoNet Aotearoa New Zealand Glossary of Data-related terms\u0026rdquo; (Abbott et al., 2021).\u003c/p\u003e \u003cp\u003eGeoNet volcano, earthquake, landslide and tsunami monitoring data comes from different scientific disciplines. These data are archived and disseminated to end users with access mechanisms and formats that are domain-specific, as this allows a full representation of the data and associated metadata. Experts in each of these fields can leverage existing tools and resources to retrieve, process and interpret the data, and will have the flexibility to tailor the processing to the specific needs of their analysis. This works well for specialised users who focus and use data from a specific domain in which they are experts but can prevent multidisciplinary studies or require collaboration across experts from multiple domains to obtain a full picture of observations from different sensors installed at a volcano.\u003c/p\u003e \u003cp\u003eThe data recording, collection, processing and dissemination infrastructure that GeoNet operates provides data from several disciplines to scientists and monitoring groups. When possible, data users would benefit from an analysis ready, easy to use and common data and metadata format disseminated through a common platform. Disciplines used in volcanology traditionally suffer from a lack of common and standardized data formats and access mechanisms for volcano data. Monitoring agencies and researchers spend a significant amount of time in searching, downloading, validating, processing and cleansing the data before they can start their analysis and interpretation.\u003c/p\u003e \u003cp\u003eLowenstern et al. (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2022\u003c/span\u003e), in their guidelines and recommendations for volcano observatory operations during crises, point out that, during a crisis, \u0026ldquo;it is fundamental to process and visualize scientific data in an integrated manner, over the same timeline and with common axes to integrate data collected at different frequencies\u0026rdquo; and that \u0026ldquo;ideally these systems incorporate multidisciplinary data that can be used by a broad range of observatory scientists who participate in analysis and evaluation of insightful correlations\u0026rdquo;. When such integrated systems are available, often they are not accessible by scientists that are outside the monitoring agency.\u003c/p\u003e \u003cp\u003eInitiatives such as WOVOdat (\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.wovodat.org/\u003c/span\u003e\u003cspan address=\"https://www.wovodat.org/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e, Venezky and Newhall, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e2007\u003c/span\u003e) from the World Organization of Volcano Observatories or the Volcano Observation Thematic Core Service of the European Plate Observing System (EPOS TCS VO, Puglisi et al., \u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e2022\u003c/span\u003e), are trying to encompass this issue, and provide free and open access to a series of data submitted by volcano observatories around the world. But, to the best of our knowledge, individual observatories and scientists wanting to monitor and research volcanoes in their country generally use a diverse range of tools, databases and access mechanisms to collate and analyse all data they have available.\u003c/p\u003e \u003cp\u003eTo facilitate interdisciplinary and open data access for Aotearoa New Zealand volcanoes, and following FAIR principles (Wilkinson et al., \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2016\u003c/span\u003e), we developed a common platform called Tilde (Time Lapse Data Entries) to ingest, store, disseminate and visualize volcano data collected by the continuous GeoNet sensor network and the VMG. We will introduce and describe the Tilde storage system and associated ingestion and access applications, using examples of volcano monitoring data. It is important to note that even though Tilde is the primary access mechanism for GeoNet volcano data, it also provides data for tsunami and landslide monitoring. We want to expose Tilde to volcanologists who might want access GeoNet data, and to those who might develop or need to understand similar systems.\u003c/p\u003e"},{"header":"Solution Design and Requirements","content":"\u003cp\u003eVolcano monitoring data have been collected by GNS Science and archived and disseminated by the GeoNet programme since its inception in 2001 (Hanson et al., this volume). Those are continuously recorded and automatically collected or manually sampled and ingested in the GeoNet data ecosystem at discrete time intervals.\u003c/p\u003e \u003cp\u003eThere are differences between how automatically and manually collected data are dealt with. Except for seismic and GNSS data, all GeoNet volcano monitoring data typically have a recording rate of once per minute or longer and are defined as \u0026ldquo;low-rate data\u0026rdquo;. Raw and processed data from continuous seismic, acoustic and geodetic stations are used for monitoring volcano and other hazards and are archived and disseminated following community standards. Data that are specific to volcano monitoring have not always been treated in a consistent way and often lack from a widely shared community standard. GeoNet recently developed systems to automatically collect ScanDOAS (Mazot et al., \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e2017\u003c/span\u003e; GNS Science, 2022) and volcano environmental monitoring data for fumaroles, springs and lakes (GeoNet, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2023\u003c/span\u003e), and these are treated like other automatically collected data. For discrete datasets, the VMG manually sample and analyse volcano features, generating a wide range of observations that are typically handled on spreadsheets and curated and archived on GNS premises. GeoNet will then collate the analysis results and distribute those to all data users. The multiparametric observations collected at Aotearoa New Zealand active volcanoes are a fundamental tools for the VMG, and it is evident that an application that allow an integrated view of multidisciplinary data was of critical importance for the GeoNet programme and the VMG.\u003c/p\u003e \u003cp\u003eFollowing the FAIR principles for scientific data (Wilkinson et al., \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2016\u003c/span\u003e) and leveraging the open access and CC-by license attributed to all GeoNet data (Hanson et al., this volume), we designed Tilde to improve findability, accessibility and interoperability of volcano and other low-rate time series data.\u003c/p\u003e \u003cp\u003eWe list below some of the design principles, but it shall be noted that the implementation of some of the Tilde components has been largely influenced by the earliest timeseries datasets that went into the system. As some of the low-rate time series data are costly to reproduce and, in some cases, (especially for manually collected volcano) the raw data are perishable, time series data needed to be archived to ensure data longevity and minimize the risk of data loss. The chosen data distribution format needed to be universally comprehensible, consistent and suitable across diverse domains. While Tilde was not designed as a metadata delivery application, serving a simplified description for each time series was developed for users who may not possess expertise in all relevant disciplines and to provide a common metadata view to programmatic access.\u003c/p\u003e \u003cp\u003eDuring the development phase of Tilde, findability and accessibility of GeoNet time series data were at the forefront of our design philosophy. The ability to provide a system to integrate and disseminate data from different disciplines was important to support and facilitate volcano monitoring.\u003c/p\u003e \u003cp\u003eA simple and intuitive data discovery solution, with user-friendly browsing features and a common perspective across different domains, was strongly required by our data users. This was one of the main drivers of designing a system that could support a user friendly and flexible Graphical User Interface (GUI) and facilitate data discovery and exploration. In fact, the wealth of GeoNet volcano monitoring data that has been historically not easy to explore and access (Hanson et al., this volume). We followed User Centred Design principles (UCD), and designed the Tilde GUI in 3 key phases: 1) a research discovery phase, during which we gathered requirements from key user groups to understand what the priority features for the GUI were; 2) a prototype design phase, during which we iterated on different design solutions to find the optimal design that worked best for the majority of end users and did not create undue load on the Tilde system; 3) a testing phase, during which we conducted one to one user testing with selected users to ensure the GUI performed as it should and find any bugs that may be present. The GUI prototype design phase ensured we had all the elements needed to meet the requirements of our end users.\u003c/p\u003e \u003cp\u003eDesign requirements also included considerations about the IT infrastructure that the Tilde system was going to rely on. Tilde backend code and infrastructure must be maintained and enhanced within GeoNet standard practices. Because GeoNet handles multi-hazard data, with a variety of types and processing systems, our preference to design a common system for time series data was to develop an in-house solution, using program languages and IT infrastructures common to other GeoNet systems. Sustainability considerations also included the ongoing cost of maintaining required storage or databases in GeoNet\u0026rsquo;s preferred cloud storage environment.\u003c/p\u003e \u003cp\u003eTilde was developed by a team composed of software developers, IT engineers, subject matter experts, system architects and user experience designers. An extended consultation with end users, including the VMG and technical experts gathered functional and non-functional requirements.\u003c/p\u003e"},{"header":"Tilde solution","content":"\u003cp\u003e\u003cstrong\u003eDesign of logical model\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe design of Tilde started with developing a domain model that could be used to describe all concepts pertinent to the GeoNet data use cases. While doing that, we explored open standards and vocabularies used to describe time series data.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe Open Geospatial Consortium (OGC) is an international consortium that develops and publishes standards for geospatial and location-based services. OGC developed the TimeSeriesML (https://www.ogc.org/standard/tsml/) and SensorML (https://www.ogc.org/standard/sensorml/) standards. We explored both, but unfortunately, we were not able to fully model our metadata requirements using TimeseriesML (too generic) or SensorML (too sensor oriented), to fully comply to all our requirements. The cost and skillsets required to develop an OGC compliant systems were not available to us within a workable timeframe.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe WOVOdat vocabulary and schema (Venezky and Newhall, 2007) was, on the other hand, too specific to volcano data for our multi-hazard requirements of Tilde and we were not able to utilize the WOVOdat vocabulary and schema either to cover all the Tilde use cases. Furthermore, the VMG, our largest volcano data user, does not use WOVOdat vocabulary and schema so trying to incorporate parts of that into how Tilde was used for volcano data did not offer any advantages.\u003c/p\u003e\n\u003cp\u003eWe opted for an in-house developed logical model and a vocabulary. The model has been designed to rationalize, map out and name all common concepts we needed to describe GeoNet time series data and how their entities are related. There are benefits and drawbacks to this decision, which we will touch on later.\u003c/p\u003e\n\u003cp\u003eThe Tilde logical model is based on the concepts of domains, series, \u0026ldquo;data objects\u0026rdquo; and their attributes (Fig. 1). A \u0026ldquo;\u003cem\u003eseries\u003c/em\u003e\u0026rdquo; is composed by all observations recorded by a sensor through time and processed to obtain measurements of a specific parameter. Each data object is used to describe individual observations at a specific point in time, and includes a timestamp, an observation value, an observation error and a quality control flag. While the quality control flag is available, we have not yet begun to use it, though works is underway to do this. Different series are grouped under the same \u003cem\u003edomain\u003c/em\u003e. A \u003cem\u003edomain\u003c/em\u003e is directly related to the type of discipline and sensor used to record the data object (for example environmental sensor). A series is uniquely identified by a number of attributes that include: a) the \u003cem\u003estation\u003c/em\u003e identifier, where the sensor is installed or manual observations is made; b) a \u003cem\u003esensor code\u003c/em\u003e identifier, used to distinguish between different sensors at the same location, or to differentiate changes in the same type of sensor through times, when those changes are significant and might impact the time series; c) a series \u003cem\u003ename\u003c/em\u003e, used to describe what is measured by the data object (for example water temperature); d) a \u003cem\u003emethod\u003c/em\u003e, used to describe the methodology used to collect the measurements or their time resolution, for example 15 minutes sampling rate, or flyspec collection system; e) an \u003cem\u003easpect\u003c/em\u003e, used to differentiate between features of the same time series, if necessary to further disambiguate the time series.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAdditional attributes are provided to describe the station metadata, and properties of all data objects in a series. Figure 1 provide a simplified snapshot of the Tilde logical model and some additional properties. A detailed and complete description of all properties is provided in the API Tilde documentation (GeoNet, 2022).\u003c/p\u003e\n\u003cp\u003eFigures 2 and 3 provides some real-world examples of how the logical model has been applied to two types of volcano monitoring time series. We provide one example for continuous data and one example for manually collected data to show what are the fundamental entities used to identify a time series, and how they can be applied for different scenarios.\u003c/p\u003e\n\u003cp\u003eFigure 2 illustrates entities of a timeseries that is generated by a datalogger measuring the temperature of a fumarole. The datalogger is part of the \u0026ldquo;envirosensor\u0026rdquo; \u003cem\u003edomain\u003c/em\u003e (GeoNet, 2023; GNS Science, 2018). The datalogger is installed at a \u003cem\u003estation\u003c/em\u003e, and the sensors in the nearby fumarole. The datalogger will record a snapshot of the temperature every 10 minutes and a maximum and minimum value over a 10-minutes interval. Different sensors are identified with the \u003cem\u003esensor code\u003c/em\u003e attribute. Each sensor will thus generate a uniquely identifiable time series, that can be grouped under the same \u003cem\u003estation\u003c/em\u003e and \u003cem\u003ename\u003c/em\u003e. The same station often has other types of sensors measuring different environmental observations.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFigure 3 illustrates a similar use case, but for the manually collected data \u003cem\u003edomain\u003c/em\u003e (or \u0026ldquo;manualcollect\u0026rdquo;; GNS Science, 1954). In this case, the sensor code attribute will be used to distinguish \u0026ldquo;spring-temperature\u0026rdquo; observations taken at slightly different points in the same area (\u003cem\u003estation\u003c/em\u003e), for example spring source and spring outflow, with a thermocouple sensor. For \u0026ldquo;manualcollect\u0026rdquo; timeseries, the \u003cem\u003easpect\u003c/em\u003e attribute is rarely used, but can differentiate observations derived from multiple samples at one time from the same feature, for example chloride concentration from spring water where all samples and analyses provide slightly different but equally valid values.\u003c/p\u003e\n\u003cp\u003eA complete description of the tilde domain model and all its entities is provided through the Tilde API documentation (GeoNet, 2022).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSystem Architecture\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA detailed description of the tilde system architecture is out of scope of this paper, but a simplified overview is provided in Fig. 4.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe majority of GeoNet data processing systems are running on Amazon Web Services (AWS) cloud infrastructure. In house developed software, configurations and databases are maintained on GitHub and systems integration and deployments are orchestrated by several processes and coded infrastructure. GeoNet data storage is relying on AWS Simple Storage Service (S3), and applications will generally utilize scheduled tasks or messaging systems and queues to run processes. A metadata database common to all GeoNet sensors networks (GNS Science, 2019) describes instruments and station metadata that all applications rely on.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eGeoNet data collection and processing systems separate from Tilde handle the continuous data feeds (at sampling rates that depends on the data domain) and generates time series observations. Similarly, VMG curate and process manually collected data separate from Tilde. Tilde time series metadata attributes are automatically extracted from the common metadata database (GNS Science, 2019), that is also used as primary source for systems that perform data processing.\u003c/p\u003e\n\u003cp\u003eOnce the continuous (or sampled) data processing is completed, each time series data point is formatted with a JSON structure compatible with Tilde and loaded in a S3 \u0026ldquo;spool\u0026rdquo; bucket. A Tilde ingest service then consumes the data point and associates it with the correct time series metadata by using the time series unique identifiers attributes and stores it on S3. Tilde uses the files stored on S3 as a form of database. This proves to be both cost effective and scalable (supports many concurrent users) however as time series grow many accumulated individual data points must be routinely repackaged into monthly and yearly files (a process referred to as \u0026ldquo;bundling\u0026rdquo;) to keep performance acceptable. The current design limits Tilde to time series with 15 second or greater time resolution, further evolutions of Tilde will improve on this, if there is a sensible use case.\u003c/p\u003e\n\u003cp\u003eWe anticipate that there will be future evolution of the Tilde system, and code, storage and frontend solutions. As such, Tilde has been designed to support versioning and upgrades whilst allowing continuity and migration of downstream processes. At the time of writing, we are currently at version 3 of Tilde.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eStorage\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTilde time series bundles are stored in Comma Separated Values (CSV) format, a widely used, open and flexible format. To enable machine learning applications and following recommendations for big datasets on cloud infrastructures, we adopted a self-documenting naming convention to store the timeseries bundles. Time series objects in the S3 Tilde archive follow a hive style partitioning in their name, where time series attributes are explicitly presented in the archived S3 object.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTilde bundled Time series data are made available through the AWS Open Data Program (GeoNet, 2022A) and can be accessed via the \u003cem\u003es3://geonet-open-data/time-series/tilde/\u003c/em\u003e S3 bucket. Timeseries are organized by version, domain, station, time series name, sensor code, time series parameters (method and aspect) and time.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eA time series bundle naming convention is as follows (the name is splitted below as bullet point list for readability):\u0026nbsp;\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003e\u003cem\u003etime-series/tilde/[v]/domain=\u003c/em\u003e[domainkey]\u003cem\u003e/\u003c/em\u003e\u003c/li\u003e\n \u003cli\u003e\u003cem\u003estation=\u003c/em\u003e[stationkey]\u003cem\u003e/name=\u003c/em\u003e[namekey]\u003cem\u003e/sensorcode=\u003c/em\u003e[sensorcodekey]\u003cem\u003e/\u003c/em\u003e\u003c/li\u003e\n \u003cli\u003e\u003cem\u003emethod=\u003c/em\u003e[methodkey]\u003cem\u003e/aspect=\u003c/em\u003e[aspectkey]\u003cem\u003e/\u003c/em\u003e\u003c/li\u003e\n \u003cli\u003e\u003cem\u003estart=\u003c/em\u003e[YYYY-MM-DD]\u003cem\u003e/\u003c/em\u003e\u003c/li\u003e\n \u003cli\u003e[domainkey].[stationkey].[namekey].[sensorcodekey].[methodkey].[aspectkey].[unit].[errorunit].[YYYY-MM-DD]\u003cem\u003eT00:00:00Z.csv.gz\u003c/em\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIn this naming style, \u0026ldquo;\u003cem\u003ev\u003c/em\u003e\u0026rdquo; is the Tilde system version and domain, \u003cem\u003estation\u003c/em\u003e, \u003cem\u003ename\u003c/em\u003e, \u003cem\u003esensorcode\u003c/em\u003e, \u003cem\u003emethod\u003c/em\u003e and \u003cem\u003easpect\u003c/em\u003e have been already discussed. \u0026ldquo;\u003cem\u003estart\u003c/em\u003e\u0026rdquo; is the starting point of the time window of the time series bundled. We chose to slice the data monthly as a compromise among the range of sampling rates of the timeseries types we currently handle. The remaining part of the S3 object name will repeat these information and add time series \u003cem\u003eunit\u003c/em\u003e attributes.\u003c/p\u003e\n\u003cp\u003eHive-style naming should allow programs and applications to easily select and slice time series observations of interest. The S3 \u003cem\u003egeonet-open-data\u003c/em\u003e access mechanism (GeoNet, 2022A) is the recommended one for those who want large volumes of data and is the most effective to use for bulk downloads.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eApplication Programmatic Interface\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe Tilde frontend interface is an Application Program Interface (API). By default, the Tilde API will return information or data in JavaScript Object Notation (JSON) format. JSON is a standard text-based format that represents structured data, is human readable but is oriented to programmatic use.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe Tilde API supports three endpoints that are used to query timeseries data, statistics and metadata.\u003c/p\u003e\n\u003cp\u003eThe data endpoint (GeoNet, 2022B) \u0026nbsp;is the mechanism to query time series data. The Tilde API allow flexibility in how timeseries data are queried, with options to slice by time interval, observation name, method and so on. Some of the attributes (\u003cem\u003esensorCode\u003c/em\u003e, \u003cem\u003emethod\u003c/em\u003e and \u003cem\u003easpect\u003c/em\u003e) can also be wildcarded by using the \u0026ldquo;-\u0026rdquo; symbol, allowing requestors to download all available time series for a given station and series name without having to know in advance all attributes of the data they\u0026rsquo;re interested in.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe stats endpoint (GeoNet, 2022C) is the mechanism to query time series observations statistics such as first and last observation, number of records, maximum, minimum, mean values and some other simple statistical calculations. This endpoint is useful for a quick inspection of the timeseries of interest and can be used to assess or do simple and \u0026ldquo;pre-canned\u0026rdquo; analysis of the observations.\u003c/p\u003e\n\u003cp\u003eThe dataSummary endpoint (GeoNet, 2022D) is a flexible mechanism to query and explore information about what timeseries are available. The endpoint can be used to query what \u003cem\u003edomains\u003c/em\u003e are available, \u003cem\u003estations\u003c/em\u003e available for a certain domain, \u003cem\u003esensorCodes\u003c/em\u003e available for a certain station and so on. An overview of the time interval covered, number of series, and associated stations, and more specific time interval for each time series \u003cem\u003ename\u003c/em\u003e is also provided.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eEach endpoint will return standard HTTP response status code to provide information on successful requests (HTTP status code 200), malformed requests (HTTP status code 400) and so on.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eEven if the API can serve bulk downloads and large data requests, we are encouraging those interested in large volumes of data to use the S3 stored time series bundles available on the GeoNet AWS Open data bucket (GeoNet, 2022).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eGraphical User Interface\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe Tilde graphical interface is built as a Data Discovery GUI (GeoNet, 2022E), following key requirements discovered during the design research phase. The Data Discovery GUI allows users to easily find what datasets are available in Tilde, what time periods they are available for, view the time series on an interactive graph and allow for data to be downloaded as a CSV file or in JSON format.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe elements on the Data Discovery GUI consist of 1) a data table with relevant domain and timeseries information; 2) a map with station icons and 3) a data query build element with dropdown selections to allow the user to select the time frame of interest and a data aggregation function (Fig. 5). Once the data query elements are selected, the user can then view the time series on a graph, download the data in JSON or CSV format, and view how the same request needs to be programmed to use the Tilde API in their own application. The design phase clearly indicated the need of data users interested in using the API to have a mechanism to quickly understand how an intuitive graphical selection shall be handled programmatically.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe Tilde Data Discovery GUI testing phase was crucial to uncover bugs in the design that made it difficult for user to perform tasks and request the data they were looking for. The iterative approach used to develop the Tilde GUI allows as to adjust the GUI design accordingly.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWe implemented some limitations on the amount of data that can be viewed using the GUI to optimize its performance. When a GUI plotting query is too large, a response message is provided to the user that recommends using the API or the AWS Open Data access mechanisms.\u003c/p\u003e\n\u003cp\u003eThe Tilde GUI has been designed in a modular way to allow for the creation of peril specific web-pages on the GeoNet website. This allows our developers to re-use the same elements created for the Data Discovery GUI to create dedicated data discovery pages for specific domains. Website pages dedicated to volcano monitoring will be developed in the future.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTutorials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIncreased usage of the low and medium sample rate data is at the heart of Tilde development. As such, alongside a detailed API documentation, we have developed a series of tutorials that showcase how the API can be used in python and bash codes. Those are publicly available via the GeoNet GitHub data-tutorials repository (GeoNet, 2020) under a dedicated section for Tilde. We would like to encourage anyone using these to contribute to the data tutorials repository and provide input if they wish. In this way, GeoNet is encouraging data users to also support each other.\u003c/p\u003e\n\u003cp\u003eGeoNet has recently started producing Data Blogs (https://www.geonet.org.nz/news; filter for Data Blogs) to illustrate the diversity of GeoNet data, showcase some interesting use cases and support with examples the understanding of how to interact with them. To date, several data blogs have focused on data that can be accessed via Tilde, and many of them with a volcano data focus. These include analysing some of the spring-based water height and temperature monitoring (GeoNet, 2023A), accessing manually collected data (GeoNet, 2024) and using a lake level monitoring site on Lake Taupo, installed to capture tsunami generated within the lake in Taupo caldera (GeoNet, 2023B). More data blogs will continue to be produced building on Tilde and its data delivery opportunities, among other topics.\u003c/p\u003e\n\u003cp\u003eEnd-users were at the heart of the Tilde design and implementation. \u0026nbsp; Features like the generation of an API request for a given GUI data search were added based on user feedback and significantly lower the barrier for programmatic data access for some users. And combining data access via a GUI, API and through GeoNet AWS Open Data provide for a wide swath of use cases, ranging from very specific data analyses to large machine-learning initiatives.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eTilde data harmonization, storage and dissemination solution has allowed the GNS\u0026rsquo;s GeoNet programme to provide a common platform for users to access and visualize time series data.\u003c/p\u003e \u003cp\u003eEfforts towards the development of the Tilde GUI massively improves technical and non-technical user and browsing experiences and provides easy access to the available data. Flexibility in the API endpoints allow users to interrogate all parameters available for any data stream and tweak the programmatic download when querying data for further data analysis.\u003c/p\u003e \u003cp\u003eThis system has improved the Findability and Accessibility of GeoNet volcano monitoring data (continuously and manually collected) and FAIR principles (Wilkinson et al., \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2016\u003c/span\u003e) have been used as a tool to guide our IT system design. The flexibility of the API and the inclusion of a versioning mechanism will improve the experience of users downloading the data programmatically, alongside the documentation and tutorials to illustrate how to interact with it. The hive-style name partitioning and the availability of Tilde time series on a cloud storage system (AWS S3), will hopefully act as enablers of machine learning applications in the future.\u003c/p\u003e \u003cp\u003eFor volcano monitoring data, Tilde will provide all time-series monitoring data. As of March 2024, it provides virtually all manually collected data and envirosensor data, and work to add sulphur dioxide fluxes from scanDOAS (Hanson et al., this volume) is underway. Ground deformation data from GNSS, for volcano and tectonic monitoring will be added within a year.\u003c/p\u003e \u003cp\u003eDue to lack of community standards in the geophysics and volcanology domains, and the effort required to implement and promote an OGC compliant standard, we ended up developing an in-house domain schema, not interoperable outside of GeoNet. This is not ideal, as analysis ready time series data could be potentially used by several disciplines if a common and interoperable standard was available and used. With the constant increase of open data, sensor devices and ability to easy share data globally, the geophysics community and its various disciplines should put efforts towards combining knowledge across various domains and build a common standard to share observations from a diverse range of geophysics disciplines. Initiatives such as EPOS (\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.epos-eu.org/\u003c/span\u003e\u003cspan address=\"https://www.epos-eu.org/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e), the recently formed EarthScope consortium (\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.earthscope.org/\u003c/span\u003e\u003cspan address=\"https://www.earthscope.org/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e), and the increasing focus on data interoperability in the geophysics community will hopefully bridge that gap in coming years.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgments\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eGeoNet is sponsored by the New Zealand Government through its agencies: Toka Tū Ake EQC, GNS Science Te Pū Ao, Toitū Te Whenua Land Information New Zealand (LINZ), the National Emergency Management Agency (NEMA) and the Ministry of Business, Innovation and Employment (MBIE). The authors wish to thank all colleagues that provided input and tested the system during its development, and in particular the GNS Science Volcano Monitoring Group and Tsunami teams.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis research was supported by the GeoNet programme through its funding agencies Toka Tū Ake EQC, GNS Science Te Pū Ao, Toitū Te Whenua Land Information New Zealand (LINZ), the New Zealand National Emergency Management Agency (NEMA) and the New Zealand Ministry of Business, Innovation and Employment (MBIE).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflicts of interest/Competing interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll authors declare that they have no conflicts of interest to disclose.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and material\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTilde Application Programmatic Interface is documented and available through the GeoNet website (https://www.geonet.org.nz/). Tilde data holdings are openly available under a CC-by license through the Tilde platform (https://tilde.geonet.org.nz/).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCode availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003enot applicable\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eAbbott E, GeoNet (2021) team GeoNet Aotearoa New Zealand Glossary of Data-related terms [Dataset], GNS Science, \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.21420/XQS0-0Z48\u003c/span\u003e\u003cspan address=\"10.21420/XQS0-0Z48\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLowenstern JB, Wallace K, Barsotti S et al (2022) Guidelines for volcano-observatory operations during crises: recommendations from the 2019 volcano observatory best practices meeting. J Appl Volcanol 11:3. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1186/s13617-021-00112-9\u003c/span\u003e\u003cspan address=\"10.1186/s13617-021-00112-9\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGeoNet (2020) GeoNet Data Tutorials. GeoNet repository on GitHub. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://github.com/GeoNet/data-tutorials/\u003c/span\u003e\u003cspan address=\"https://github.com/GeoNet/data-tutorials/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 30 March 2024\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGeoNet (2022) AWS Open Data Access. GeoNet website \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.geonet.org.nz/data/access/aws\u003c/span\u003e\u003cspan address=\"https://www.geonet.org.nz/data/access/aws\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 30 March 2024\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGeoNet (2022A) Tilde Application Programming Interface. GeoNet website. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://tilde.geonet.org.nz/\u003c/span\u003e\u003cspan address=\"https://tilde.geonet.org.nz/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 30 March 2024\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGeoNet (2022B) Tilde Application Programming Interface Observation Data endpoint. GeoNet website. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://tilde.geonet.org.nz/v3/api-docs/endpoint/data\u003c/span\u003e\u003cspan address=\"https://tilde.geonet.org.nz/v3/api-docs/endpoint/data\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 30 March 2024\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGeoNet (2022C) Tilde Application Programming Interface Observation Statistics endpoint. GeoNet website. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://tilde.geonet.org.nz/v3/api-docs/endpoint/stats\u003c/span\u003e\u003cspan address=\"https://tilde.geonet.org.nz/v3/api-docs/endpoint/stats\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 30 March 2024\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGeoNet (2022D) Tilde Application Programming Interface Data Summary endpoint. GeoNet website. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://tilde.geonet.org.nz/v3/api-docs/endpoint/dataSummary\u003c/span\u003e\u003cspan address=\"https://tilde.geonet.org.nz/v3/api-docs/endpoint/dataSummary\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 30 March 2024\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGeoNet (2022E) Tilde Time Series Data Discovery. GeoNet website. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://tilde.geonet.org.nz/ui/data-exploration\u003c/span\u003e\u003cspan address=\"https://tilde.geonet.org.nz/ui/data-exploration\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 30 March 2024\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGeoNet (2023) GeoNet Environmental Data. GeoNet website \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.geonet.org.nz/data/types/environmental\u003c/span\u003e\u003cspan address=\"https://www.geonet.org.nz/data/types/environmental\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 30 March 2024\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGeoNet (2023A) Using Our Own Data \u0026ndash; Hot Spring Water Level and Temperature. GeoNet website \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.geonet.org.nz/news/2YpqCnWiYCMUEkaPi9ZJj3\u003c/span\u003e\u003cspan address=\"https://www.geonet.org.nz/news/2YpqCnWiYCMUEkaPi9ZJj3\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 30 March 2024\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGeoNet (2023B) New Seismic, GNSS and Tsunami Monitoring Sites at Lake Taupo \u0026ndash; How Delta Drives GeoNet\u0026rsquo;s Data Processing and Discovery. GeoNet website \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.geonet.org.nz/news/2bY6I9tix34IANrWcI9s18\u003c/span\u003e\u003cspan address=\"https://www.geonet.org.nz/news/2bY6I9tix34IANrWcI9s18\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 30 March 2024\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGeoNet (2024) Access to Manually Collected Volcano Data is Moving from the FITS API to the Tilde API. GeoNet website \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.geonet.org.nz/news/26beUFBJGh5t03JtoWtRft\u003c/span\u003e\u003cspan address=\"https://www.geonet.org.nz/news/26beUFBJGh5t03JtoWtRft\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 30 March 2024\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGNS Science (1954) GeoNet Aotearoa New Zealand Manually Collected Volcano Data [Dataset]. GNS Science. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.21420/PSP7-KW60\u003c/span\u003e\u003cspan address=\"10.21420/PSP7-KW60\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGNS Science (2018) GeoNet Aotearoa New Zealand Automatically Collected Volcano Data [Dataset]. GNS Science. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.21420/EN0F-XY29\u003c/span\u003e\u003cspan address=\"10.21420/EN0F-XY29\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGNS Science (2019) GeoNet Aotearoa New Zealand Stations Metadata Repository [Dataset]. GNS Science, GeoNet. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.21420/0VY2-C144\u003c/span\u003e\u003cspan address=\"10.21420/0VY2-C144\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMazot A, Christenson BW, Scott BJ (2017) Mini-DOAS data processing for SO2 emission measurements and monitoring volcanic degassing on White Island (Whakaari), New Zealand. p. 668 in: IAVCEI 2017 Scientific Assembly abstracts: fostering integrative studies of volcanoes, August 14\u0026ndash;18, Portland, Oregon, U.S.A. IAVCEI\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePuglisi G, Reitano D, Spampinato L, Vogfj\u0026ouml;rd KS, Barsotti S et al (2022) The integrated multidisciplinary European volcano infrastructure: from the conception to the implementation. Ann Geophys, 65 (3), pp.DM320. ⟨10.4401/ag-8794⟩. ⟨hal-03958096⟩\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVenezky DY, Newhall CG (2007) WOVOdat design document; the schema, table descriptions, and create table statements for the database of worldwide volcanic unrest (WOVOdat Version 1.0): U.S. Geological Survey Open File Report 2007\u0026thinsp;\u0026ndash;\u0026thinsp;1117, 184 p. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://pubs.usgs.gov/of/2007/1117/\u003c/span\u003e\u003cspan address=\"http://pubs.usgs.gov/of/2007/1117/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWilkinson M, Dumontier M, Aalbersberg I et al (2016) The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3:160018. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/sdata.2016.18\u003c/span\u003e\u003cspan address=\"10.1038/sdata.2016.18\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":true,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bulletin-of-volcanology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"buvo","sideBox":"Learn more about [Bulletin of Volcanology](http://link.springer.com/journal/445)","snPcode":"445","submissionUrl":"https://www.editorialmanager.com/buvo/default2.aspx","title":"Bulletin of Volcanology","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"volcano monitoring data, time series data interoperability, data ingestion and dissemination, application programmatic interface, graphical user interface, data storage","lastPublishedDoi":"10.21203/rs.3.rs-4190192/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4190192/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTilde is a new in-house developed solution that the GNS Science\u0026rsquo;s GeoNet programme has recently developed to provide storage and access to low sample rate datasets used to monitor tsunami, landslides and volcanoes in Aotearoa New Zealand. It includes datasets covering sample rates of minutes or longer. Time series data are stored and disseminated in JSON and CSV formats, and users can access these through an Application Programming Interface (API) and a Graphical User Interface (GUI). Tilde\u0026rsquo;s GUI was created to allow technical and non-technical users easy access to the available data. The introduction of Tilde system as one of the GeoNet programme delivery channels has represented a big step forward for GeoNet\u0026rsquo;s volcano data holdings by providing a single point to access all low to medium sample rate volcano specific monitoring data. We designed the system, developed a domain model, an API, a graphical data discovery interface and associated data tutorials. This work leverages the Open by Default data policy for data generated through the GeoNet programme. This paper is intended to highlight how we made many of the key decisions that shaped Tilde system, how they were impacted by our multi-hazard monitoring requirements and how they have improved access to volcano monitoring data. We conclude with some open questions about the need of developing common standards to share analysis-ready time series data within different disciplines in volcanology and geophysics.\u003c/p\u003e","manuscriptTitle":"Tilde, an interdisciplinary data access solution serving GeoNet's volcano and other hazard data","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-04-23 12:58:50","doi":"10.21203/rs.3.rs-4190192/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Minor Revisions","date":"2024-08-20T05:11:36+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"","date":"2024-04-23T19:00:42+00:00","index":0,"fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-04-19T04:06:47+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"Bulletin of Volcanology","date":"2024-04-18T10:40:45+00:00","index":"","fulltext":""},{"type":"submitted","content":"Bulletin of Volcanology","date":"2024-03-29T22:02:03+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bulletin-of-volcanology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"buvo","sideBox":"Learn more about [Bulletin of Volcanology](http://link.springer.com/journal/445)","snPcode":"445","submissionUrl":"https://www.editorialmanager.com/buvo/default2.aspx","title":"Bulletin of Volcanology","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"2687ea36-4e2f-4143-8471-69f9f9e79904","owner":[],"postedDate":"April 23rd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2024-12-02T16:02:59+00:00","versionOfRecord":{"articleIdentity":"rs-4190192","link":"https://doi.org/10.1007/s00445-024-01786-w","journal":{"identity":"bulletin-of-volcanology","isVorOnly":false,"title":"Bulletin of Volcanology"},"publishedOn":"2024-11-28 15:57:43","publishedOnDateReadable":"November 28th, 2024"},"versionCreatedAt":"2024-04-23 12:58:50","video":"","vorDoi":"10.1007/s00445-024-01786-w","vorDoiUrl":"https://doi.org/10.1007/s00445-024-01786-w","workflowStages":[]},"version":"v1","identity":"rs-4190192","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4190192","identity":"rs-4190192","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}