Detection of gamma-ray polarization in Positron Emission Tomography (PET) is yet an unexploited feature that could be used as an additional handle to improve signal-to-background ratio in this imaging modality. The gamma polarization is related to the azimuthal angle in the Compton scattering process, so the initial correlation of polarizations translates to the correlation of azimuthal angles in true coincidence events, while it lacks in the background. We will present a comprehensive experimental study of the polarimetric performance of 5 detector configurations based on scintillator matrices and silicon photomultipliers. The modules consist of either GaGG:Ce or LYSO:Ce pixels with sizes varying from 1.9x1.9x20 mm$^3$ to 3x3x20 mm$^3$. The distinctive feature of the modules is that they can reconstruct the Compton scattering by detecting the recoil electron and the scattered gamma in a single detector layer, which simplifies extension to larger systems. We will discuss the factors driving the optimal polarimetric performance of the modules and the perspective of their application in PET.