It's possible to use the Document.Set function and get a resulting document with the wrong field replaced. After discussion with kris.brandow it seems like the problem is this line:

https://github.com/mongodb/mongo-go-driver/blob/master/bson/document.go#L259

which should be replaced with d.elems[d.index[i]] = elem