With the (partial) introduction of the readAtCluster time feature in version 4.0, we introduced the history field to entries in config.chunks. The FCV upgrade step from 3.6 to 4.0 forcefully writes the history field to all chunks in the system and bumps their version in order for the cache to be able to see the change.
This however, causes problems on upgrades, because it causes a total throw-out of the caches and also full rewrite of the local cache collections on the Shards, which in the case of many chunks can cause minutes of stall. These stalls on top of that are unpredictable and might not be related when the FCV command was run.
I am fairly certain that we can throw out the force-write of the history field and leave them to happen lazily. The code even in master is prepared to handle the missing field.
On downgrades I think it is also safe to not bother removing it. Older code will ignore it and if another upgrade is performed, there is no harm with having the values there, because they will map to a very old cluster time which will be smaller than the current one, so no harm.