Amazing discoveries by an aging storage professional catching up with BigData
In the previous post I describe a typical workflow for data-intensive science. Sizeable IT infrastructure is required to handle the data rates and the data volume from the data acquisition and fast feedback via the iterative analysis to the archive. The required infrastructure can be provided by high-performance computing (HPC), cloud computing, or a hybrid …
In the previous post I characterize data-intensive science as research and engineering efforts where the storage, the management and the analysis of acquired data requires special considerations to enable the overall scientific effort. So, what does a typically workflow in data-intensive science look like? The figure below depicts a typical workflow in data-intensive science. The …
In the previous post I claim that data-intensive science forces organizations to adopt high-performance computing (HPC). I heard the phrase “data-intensive science” the first time in 2012 when I worked with a research institution on large-scale file services for scientists. Since then I heard “data-intensive science” from multiple clients, though in 2019 “data-intensive science” has …
Since 2012 I transition from storage to high-performance computing (HPC). Today I work with many clients who need to transition from file servers and Network Attached Storage (NAS) to HPC to store and analyze acquired data effectively. In this HPC meets AI series I will introduce basic HPC concepts and show how HPC can enable …