In the field of High Energy Physics, the simulation of the interaction of particles in the material of calorimeters is a computing intensive task, even more so with complex and fined grained detectors. The complete and most accurate simulation of particle/matter interaction is primordial while calibrating and understanding the detector at the very low level, but is seldomly required at physics analysis level, once several level of detector effect smeared and hides slight imperfection in simulation. Some level of approximation in the simulation is therefore acceptable and one can move on with implementing fast simulation surrogate models which have the advantage of being less computationally intensive. Going further than parameterized simulation already implemented in HEP experiment, we present a fast simulation based on generative adversarial networks. The model is constructed from a conditional generative model of calorimeter response and a discriminative model to distinguish between generated and ground truth energy deposition, trained in adversarial manner. We use a dataset composed of the energy deposition from electron, photons, charged and neutral hadrons in a fine grained digital calorimeter. The training of these deep neural net models is quite computing intensive, even with the help of GP-GPU, and we propose a method to train them over multiple node and GPGPU using a standard message passing interface. We report on the scalings of time-to-solution. Further tuning of hyper-parameter of the models are rendered tractable and we present the physics performance of the best model obtained via a bayesian optimization using gaussian processes. We demonstrate how high-performance-computing center can be utilized to globally optimize this kind of models.