It appears to me that if we run the maps for a longer time, the already-converged results (with a smaller number of atoms in the cells) becomes diverged. In other words, the message initially is "Among structures of known energy, true and predicted ground states agree." However, after a while, it becomes "Among structures of known energy, true ground states differ from fitted ground states"
Then the question is when to stop running maps? Would the eci file saved from the converged results still be useful? Thank you.
I think on top of a converged convex hull, you either need to have prior knowledge, or have to wait until a large number of active sites are considered in the supercell, e.g. 16.
It can happen that for a small number of structures, maps seems to have converged and everything looks good but that, for a larger set, some problems occur. The latter problems indicate either that
the initial small cluster expansion was obtained "by luck" - the real energetics of the system is more complex and maps is discovering that at a later stage.
the more complex cluster expansion could be momentarily showing problems that will eventually disappear as more structures are added.
the system exhibit mechanical instabilities. Initially, by luck, none of the structures were unstable but then unstable ones occurred, which makes the fit very difficult. You can investigate this with the checkrelax command and see if relaxations are large (>0.07 or so)
A new rigorous solution to this problem can be found in https://dx.doi.org/10.1038/ncomms8559