The church is first mentioned in 1461 as being on the Green and was sometimes called the Green Church. The dedication to St. James is known by 1505. The church may occupy the site of the hospital chapel of St. James and St. Denis, which had gone by 1338. Originally the church was a chapel in Bishop's Cannings parish but it had its own graveyard for burials by 1505 and christenings and marriages were taking place by 1572. The church seems to have been built in the early 15th century with a chancel, nave with continuous aisles, a south porch and a west tower. There were some lancet windows of the 13th century that possibly came from the original chapel.
Action during the Civil War resulted in cannonballs and shot hitting the church, the marks of which can still be seen on the tower. To provide accommodation for the growing congregation in this part of Devizes the church was rebuilt in 1831-2 to the design of John Peniston. Some of the original stone was reused. The church now consists of a chancel of one bay, an aisled nave of three bays with a west gallery and the 15th century tower, part of which serves as a porch. The churchyard was enlarged c.1844 and closed for burials in 1876. The registers dating from 1572, other than those in current use, are held in the Wiltshire & Swindon History Centre in Chippenham.