ADCME is a novel computational framework to solve inverse problems involving physical simulations and deep neural networks (DNNs). By describing physical laws with partial differential equations (PDEs) and substituting unknown components with DNNs, we preserve the physics to the largest extent while leveraging DNNs for data driven modeling. To train the DNNs within a physical system, ADCME expresses both numerical simulations (e.g., finite element method) and DNNs as computational graphs and calculates the gradients using reverse-mode automatic differentiation. We have built a system of re-usable and flexible numerical simulation operators that support gradient-backpropagation for many engineering applications, such as seismic inversion, constitutive modeling, Navier-Stokes equations, etc. ADCME also provides a computational model for conducting large-scale inverse modeling using MPI, and has been deployed across thousands of cores. The ADCME software is open-sourced and available at https://github.com/kailaix/ADCME.jl.