We present a detailed study on Variational Autoencoders (VAEs) performing in anomalous jet tagging. By taking in low-level jet constituents' information, and only training with background jets in an unsupervised manner, the VAE is able to encode important information for reconstructing jets, while learning an expressive posterior distribution in the latent space. The encoder (inference) and decoder (generation) can be used together or seperately to identify out-of-distribution anomalous jets. We employed different techniques to regularize the latent representation, and show how the behavior changes. When using VAE as anomaly detector, we present two approaches to detect anomalies: directly comparing in input space or, instead, working in latent space. Results of tagging performance for different jet types and full kinematic range are shown. In addition, we also study a few tricks to make VAE more sensitive to anomalies.