2014 Rice Oil & Gas HPC

Workload and distributed resource management systems (often also referred to in a less descriptive way as batch schedulers) have been used as part of the of the typical HPC computing infrastructure in the Oil and Gas industry for more than a decade. So, certainly, all problems are solved and these schedulers are legacy, right? Nothing could be farther from the truth! Get the workload and resource management part wrong and all your finesse applied around application tuning, server/storage/network architecture or energy saving is a waste of effort and money to a large degree. Why is workload and resource management so crucial and why now more than ever? In this presentation we will discuss why workload and resource management is the conveyer belt which drives production in an HPC data center and what cost and productivity impacts are incurred when the workload and resource management system slows operation down or even brings it to a halt. We will provide concrete examples from practice how choices concerning the workload and resource management system and the infrastructure around it have immediate and considerable impact on time-to-results and throughput. Aspects being discussed in this context will be the selection of tools, dependencies on networking and storage, configuration and policy set-up best practices, considerations on application integration plus the utilization of accounting, reporting and analytics tools for optimizing the HPC infrastructure. Furthermore we will investigate how recent and emerging trends in the evolution of HPC infrastructure components present new challenges to workload and resource management systems and how such trends need to be taken into account when planning infrastructure upgrades. We will take a look at the growing complexity from increasingly heterogenous server hardware (multi-socket, multi-core, many core, GPUs, NUMA - to just list a few key words) and what is required to exploit such technologies to meet the scalability and throughput goals anticipated for the next generation Oil and Gas data centers. We will also showcase how energy saving efforts revolve around the workload and resource management system. [Remark: it is our intention that this presentation will be held jointly with a representative of one of our key Oil & Gas customers, namely from either Saudi Aramco, BP or PGS. This speaking opportunity at the 2014 Rice Oil & Gas Workshop was brought to our attention with too short notice to be able to get any of our customer liaisons go through their respective company internal approval process to be able to commit to participate. We therefore had to submit the paper as it stands just with a speaker from Univa.]