This study describes the design and construction of a Chemical Vapor Deposit (CVD) diamond detector used to detect neutron radiations. The use of a neutron detector in a thin CVD diamond film has many benefits, including radiation hardening, fissile-material free, low gamma susceptibility, compact and solid state, spectroscopic, both thermal and quick neutron detection, especially when external "blankets" are used to turn neutrons into visible charge particles. As Gd and Ag neutron detectors, thermal and fast neutron detectors have been developed. In addition, a simultaneous detector capable of detecting both thermal and rapid neutron radiations was built. The thin film deposition of Gd and Ag electrodes was calculated to be 4 m and 150 nm, respectively, based on the geometry and measurement.The performance testing to characterize the response of CVD diamond detectors has been carried out at the KIGAMS MC-50 Cyclotron having a 30 MeV proton energy and 10 µA current, which is an accelerator based neutron sources with the high neutron flux about 1x10$^4$ ~1x10$^6$ n.cm$^-$$^2$.s$^-$$^1$. The detector counting efficiency and energy resolution were accordingly derived as a function of the thickness of the $^6$LiF and CVD diamond layers, both for thermal and fast neutrons, thus allowing us to choose the optimum detector design for any particular application. Comparison with experimental results is also reported.