Standing Committee Meeting Report Data Services

October 2018

Action Items from the Fall 2017 DSSC Meeting

Action Item F2017:2 Previous Action Item F016:1 Follow up with Greg on archiving historical data, often from other agencies. Are there potential sources of funding within other parts of NSF?
Responsible: Stump will assume full responsibility but the point of contact should move from Anderson to Benoit.
Responsible: Stump

Action Items from the Spring 2018 DSSC Meeting

Action Item S2018:2&11 Propose to BoD that there is a need for a discussion related to financial support for Data Services from other government agencies.   A good starting point would be NSF, USGS, and IRIS.
Responsible: Stump

Action Item S2018:4 Revisit the Long-Term Temporary Data policy specifically related to the cost of absorbing Large-N data sets.
Responsible: Ahern and DMC Staff

Action Item S2018:7 Need to further research a method to affordably manage Large-N sized data sets. Consider lower cost storage systems and understand the impact that it might have on current DMC services.
Responsible: Ahern and DMC Staff

Action Item S2018:9 DSSC supports the concept of writing a cover letter for the letters of support for Data workshops.  Include letter of support from a previous participant of a workshop indicating the helpfulness of the workshops for regional data sharing. Circulate through the DSSC and then forward this to the BoD.Return to this topic at the Fall Meeting.
Responsible: Stump and Ahern

Action Items from the Fall 2018 DSSC Meeting

F2018:1  Invite Lorraine Hwang and Paul Richards to Spring 2019 Meeting of the DSSC to discuss next steps towards moving historical data rescue forward.
Responsible: Stump

F2018:2 Develop a community letter supporting archiving historical data at the National Archives
Responsible: Dave Wilson

F2018:3 Distribute the priorities for Data Services to the IRIS Board
Responsible: Stump/Ahern

F2018:4 Decision: DSSC strongly supports two face-to-face meetings of the DSSC. Ahern to build Sage2 YR2 budget with support for two meetings
Responsible: Ahern

F2018:5 Develop video tutorials for MUSTANG clients and other tools related to data access. Perhaps consider engaging a UW student intern​
Responsible: Ahern

F2018:6 DSSC chair to contact PASSCAL SC chair about delinquent data sets from PASSCAL PIs
Responsible: Stump

F2018:7  Prioritize any potential SAGE II Yr1 carryover from IS and DS for Matlab interfaces to MUSTANG metrics: Take to COCOM to coordinate with IS
Responsible: Stump/Ahern

F2018:8 Take Data workshop letter to BoD and COCOM and then send to NSF
Responsible: Stump

F2018:9 Keep DSSC updated on Mid-Scale Infrastructure approach for support of the CDF proposal for shared infrastructure.
Responsible: Ahern

F2018:10 Decision: Surface Radiation Pattern product was approved by the DSSC.

F2018:11 Take need for better support for PIs and their workflows to get data into the DMC whether it be in SEED or PH5. Especially the issue of non-IRIS nodal systems. Take to COCOM
Responsible: Stump/Ahern

F2018:12 Take opt out approach of data not being at the DMC to the FDSN.   Any data center has the right to not have data stored at the IRIS data center.  Try to get a level of service agreement worked out in FDSN WGIII.  Continue this action through at least the next DSSC meeting
Decision: DSSC concurs with the above statement and supports the general concept of Federation.

F2018:13 Need one new member of DSSC.  Send list of potential new members to COCOM.

F2018:14 Decision: Next meeting in Seattle (end of Feb or beginning March). Ahern to circulate a doodle poll
Responsible: Ahern

F2018:15 Invite Chuck Meertens, BOD Liaison  and GDSAC chair to the spring meeting of the DSSC.
Responsible: Stump/Ahern

F2018:16 Take the updated “Data Acceptance Policy for Datasets from Temporary Deployments” to the Cocom for discussion and the BoD for approval.

F2018:17 After BoD approval of the “Data Acceptance Policy for Datasets from Temporary Deployments” Moira will talk to individuals at collaborating institutions for their views.

F2018:18 Let CoCom/BoD know that the DSSC has concerns about meeting quorum with only 7 members on the standing committee.

Brief Meeting Summary

The key items identified for this meeting were identified by DSSC Chair, Brian Stump as

  • New Policy for Temporary Data Sets needs robust discussion (C1)
  • IRIS Data workshops letter to NSF (C2)
  • Information on Director Search (C3)
  • NSF has asked for an external review of IRIS and UNAVCO Data Services
  • Recommendations for new committee members (E4)

Minutes and Action items from the Spring 2018 DSSC meeting were updated and approved. Recent changes in staffing were identified including the retirement of two long term Data Control Technicians whose duties are being more automated now with the hire of Tim Ronan as a Data Engineer. Resulting staff reduction of 1 FTE. Additionally, the assistant Systems Administrator has been replaced with a new Systems Administrator. DMC staff is now reduced from 27 FTE’s at the beginning of SAGE to 21 FTEs currently.

Progress was reported on the development of two new formats, StationXML, a replacement for dataless SEED and a new replacement for miniSeed is also being developed. Efforts in this area will be coordinated with our partner’s in the FDSN.

The availability of Rover, a new robust method of delivering data through web services was announced. It should enter service in the 1Q2019.

DSSC expressed a concern with the amount of delinquent data from some PIs. DSSC wishes to coordinate with the PASSCAL Standing Committee to see if this can be addressed.

The progress of installing and using the newer Supermicro RAID systems for Large-N data was discussed. Initial performance seems very good with these new systems. The new DSSC policy for managing data from temporary deployments was discussed at length and shortly after the meeting many modifications were made to the new policy that went forward to the IRIS CoCom.

The Surface Wave Radiation Pattern product developed from an intern at Northwestern was presented and discussed by the DSSC.

Progress in the search for a new Director of Data Services was presented to the DSSC by Brian Stump who is chairing the search committee.

DSSC was supportive of blocking SWARM clients that were trying to pull near real time data through web services as an alternative exists by using SeedLink in the SWARM application. The inappropriate use of SWARM was impacting the reliability of web services for everyone through this misuse.

DSSC was updated on the need to relocate the primary DMC staff offices due to the upcoming sale of the Wells Fargo bank building.

Ahern updated the DSSC with the progress of the Federated System, The DSSC recommended that as long as data are reliably available through a federated center other than IRIS it would be acceptable to remove the data from the DMC. [This was also discussed at the later Board meeting and with the caveat about reliability the BoD agreed the data on a case by case basis could be removed.

The status of unfinished tasks and carryover at the end of SASGE YR 5 was updated. DS basically had no funds remaining after this exercise.

Discussions regarding the future coordination between IRIS and UNAVCO Data Services took place.