The traditional finite element method needs to supplement a stabilization scheme to simulate Oldroyd-B viscoelastic flows. To alleviate this issue, a unified discontinuous Galerkin finite element framework based on unstructured grids is proposed in this paper. The system contains two key points: one is using the IPDG (interior penalty discontinuous Galerkin) method to discretize mass and momen-tum equations, and the other is employing the RKDG (Runge-Kutta DG) method to solve the Oldroyd-B constitutive equation. Simulation results reveal the intrinsic characteristics of non-Newtonian viscoelastic fluids and indicate that the approach can effectively overcome the drawback of the traditional finite element method, which redundantly introduces stabilization process in the method. Moreover, these results substantiate that the proposed method is simple to implement, has high accuracy and can be used to simulate complex viscoelastic flows with stress singularity.