Welcome to the Earth Science Information Partners (ESIP) 2018 Summer Meeting! The 2018 theme is Realizing the Socioeconomic Value of Data. The theme is based on one of the goals in the 2015 - 2020 ESIP Strategic Plan, which provides a framework for ESIP’s activities over the next three years.
All Presentations are being added to a Google Folder temporarily and then will be moved to FigShare and linked to the sessions here.
TaskAPI is a workflow platform and DSL (Domain Specific Language) that provides automatic horizontal and vertical scaling of multi-language data-intensive scientific software systems using a functionally declarative workflow paradigm. TaskAPI is capable of quickly wrapping legacy systems, provides structured guidance for best-practices in continued or new development via its JSON DSL, and automatically provides system components with a unified, straightforward API for centralized logging, job and task killing, and configurable property use.
TaskAPI was developed to serve as the backbone for the reengineered US ASOS Ingest software system and exists as its own distributable package for use by other large polyglot systems.
This session will begin by providing a broad overview (surface skim) of the TaskAPI platform, including motivation, capabilities, current and potential use cases, and design and performance characteristics.
The summary will lead into a more detailed look at the TaskAPI structure, including the DSL setup, workflow branching, task types, multi-language parallelization techniques in Java, C, and Fortran, and current and planned language support (Python via Jep, Clojure, Scala via drivers), and other features (Kafka messaging queues).
After we thoroughly explain the system and its capabilities, we will deep dive into a live example of TaskAPI as it was implemented in ASOS, examining real-life challenges we faced and how to think about and implement best use practices.
Finally, we will assist session attendees and participants in determining if this system could serve their own projects, provide assistance in TaskAPI download and setup, and solicit feature requests and needs.
