Better place to remove the draft callback, where we hold the mutex