Certifying the data recorded by the Compact Muon Solenoid (CMS) experiment at CERN which is usable for publication of physics results is a crucial and onerous task. Anomalies caused by detector malfunctioning or sub-optimal data processing are difficult to enumerate a priori and occur rarely, making it difficult to use classical supervised classification. We base out prototype towards the automation of such procedure on a semi-supervised approach using deep autoencoders. We demonstrate the ability of the model to detect anomalies with high accuracy, when compared against the outcome of the fully supervised methods. We show that the model has great interpretability of the results, ascribing the origin of the problems in the data to a specific sub-detector or physics object. Finally, we tailor the approach with a systematic method for feature filtering and address the issue of feature dependency on LHC beam intensity.