We are interested in the convergence and the local regularity of the lacunary Fourier series $F_s(x) =\sum_{n=1}^{+\infty} e^{ 2i\pi n ^2 x}{ n^s}$. In the 1850's, Riemann introduced the series $F_2$ as a possible example of nowhere differentiable function, and the study of this function has drawn the interest of many mathematicians since then. We focus on the case when $1/2 < s \leq 1$, and we prove that $F_s(x)$ converges when $x$ satisfies a Diophantine condition. We also study the $L^2$-local regularity of $F_s$, proving that the local $L^2$-norm of $F_s$ around a point $x$ behaves differently around different $x$, according again to Diophantine conditions on $x$.