Asap update

Dear Ryan and co.

Just in case you are using Asap in your OpenKIM workflow: I have squashed a few bugs related to OpenKIM models (it failed to calculate virials, and it failed to properly detect if you tried to use a model with an unsupported chemical element). I also fixed a few compilation issues on picky compilers. So if you build from the openKimTwo branch on gitlab, you should update.

BTW, the EMT model is almost ready. Still needs some code cleanup.

Best regards

Jakob

Dear Jakob,

That is great! Thanks for the work and the update.

By the way, we are preparing for another beta replease (kim-api-v2-2.0.0-beta.3) in the next few days. The updates for this release are (essentiall) all aready in the 'devel-v2' branch of the kim-api github repo.

Significant changes/additions include:

* A new KIM_RoutineNames enumeration providing a run-time discoverable interface for the list of "top-level" model routines (Create, ComputeArgumentsCreate, Compute, etc.) defined by the kim api.

* A new IsRoutinePresent() mechanism that Simulators need to use in order to determine if they are fully compatible with a particular model.

* A change in the ModelCreate interface from individual calls to set routine pointer (SetComputePointer(), SetDestroyPointer(), etc.) to a unified call for each ModelRoutine: SetRoutinePointer().

* Two new top-level routines added:

   - WriteParameterizedModel(): This provides a mechanism whereby a ModelDriver
     may be asked to write out its current parameters (which may have been
     adjusted from their default values by the simulator's use of SetParameter()
     and ClearThenRefresh()).

   - Extension(): This proivides a very general way to extend the KIM API. A
     Simulator and Model must both know about and agree on an extension data
     strucuture. Once this requirement is met, then they may communicate in a
     standard fashion that allows for any task to be performed.

* Some minor new functionality relateing to the KIM API logging mechanism has been added to the KIM::Log class: (PushDefaultVerbosity() and PopDefaultVerbosity()).

The examples in the kim-api repo have been updated and adjusted to provide usage examples of the significant additions/changes described above.

Cheers,

Ryan

[ ... ]

By the way, we are preparing for another beta replease (kim-api-v2-2.0.0-beta.3) in the next few days. The updates for this release are (essentiall) all aready in the 'devel-v2' branch of the kim-api github repo.

Great. I will wait for the release to avoid double work.

Significant changes/additions include:

* A new KIM_RoutineNames enumeration providing a run-time discoverable interface for the list of "top-level" model routines (Create, ComputeArgumentsCreate, Compute, etc.) defined by the kim api.

I don’t really see the point of this, so I am probably completely misunderstanding it. Could you elaborate what this is doing?

* A new IsRoutinePresent() mechanism that Simulators need to use in order to determine if they are fully compatible with a particular model.

I am not sure what a model Routine is. Is it stuff like Compute etc? If so, why is this needed, don’t you already get a NULL pointer if you try to get a routine that the model does not provide?

* A change in the ModelCreate interface from individual calls to set routine pointer (SetComputePointer(), SetDestroyPointer(), etc.) to a unified call for each ModelRoutine: SetRoutinePointer().

OK. Will a ComputeArgumentsDestroy then be needed? My current model does not set any, mainly because I don’t think I need it, but also because I was not able to work out what the prototype should be.

* Two new top-level routines added:

- WriteParameterizedModel(): This provides a mechanism whereby a ModelDriver
   may be asked to write out its current parameters (which may have been
   adjusted from their default values by the simulator's use of SetParameter()
   and ClearThenRefresh()).

So this returns a big string? This makes a lot of sense, since it can be formatted in a more clever way that what the Simulator could do, as the simulator does not know what the parameters mean (nor does it know their “natural” shape, see below).

- Extension(): This proivides a very general way to extend the KIM API. A
   Simulator and Model must both know about and agree on an extension data
   strucuture. Once this requirement is met, then they may communicate in a
   standard fashion that allows for any task to be performed.

Cool, but a bit dangerous, since it might cause models to lose their generality.

May I suggest an optional feature: GetParameterShapeHint(int const parameterIndex) returning a string giving information about the shape of the parameter array. If not supported, it returns the empty string, and the simulator only knows the information returned by GetParameterMetadata(). The list of possible return values can be extended, but includes

“OneDimensional”. A 1-D array, the size is known from GetParameterMetadata().
“Square” An N*N matrix, N can be deducted from the size of the array (N^2).
“Cube” An N*N*N array, the size is N^3.
“UpperTriangular” An upper triangular matrix of size N*N, where the actual size of the array is N*(N-1)/2
“UpperTriangularWithZeros” Like “Square”, but elements below the diagonal are ignored.
“LowerTriangular”
“LowerTriangularWithZeros"
“(2, 43, 32, 10)” Actual dimensions of the array.
“” None of the above, or a scalar.

* Some minor new functionality relateing to the KIM API logging mechanism has been added to the KIM::Log class: (PushDefaultVerbosity() and PopDefaultVerbosity()).

Some kind of documentation on how to use the logging would be nice. Also, is there a way for the simulator to discover the errors that the model may have logged, so they may be passed on to the user?

The examples in the kim-api repo have been updated and adjusted to provide usage examples of the significant additions/changes described above.

Nice! The examples are very useful indeed.

Best regards

Jakob

I actually have one more feature request: Some way for a Simulator to discover a list of installed models. Perhaps just a command to kim-api-v2-collection-management, where it lists the models without the human-readable formatting.

Best regards

Jakob

  [ ... ]

By the way, we are preparing for another beta replease (kim-api-v2-2.0.0-beta.3) in the next few days. The updates for this release are (essentiall) all aready in the 'devel-v2' branch of the kim-api github repo.

Great. I will wait for the release to avoid double work.

Significant changes/additions include:

* A new KIM_RoutineNames enumeration providing a run-time discoverable interface for the list of "top-level" model routines (Create, ComputeArgumentsCreate, Compute, etc.) defined by the kim api.

I don’t really see the point of this, so I am probably completely misunderstanding it. Could you elaborate what this is doing?

This is needed for use with IsRoutinePresent below.

* A new IsRoutinePresent() mechanism that Simulators need to use in order to determine if they are fully compatible with a particular model.

I am not sure what a model Routine is. Is it stuff like Compute etc? If so, why is this needed, don’t you already get a NULL pointer if you try to get a routine that the model does not provide?

The main need for this is to support backward compatibility in the case where new features are added to the kim-api.

Yes, these are the routines:

Compute, WriteParameterizedModel, ComputeArgumentsCreate/Destroy, Extension, etc.

Using this mechanism, a simulator can discover if a model it wants to use requires that the simulator employ features if the api that it does not know anything about. In that case the simulator can fail gracefully.

Suppose, as a hypothetical, in v2.2.0 we add a ChargeCompute() routine and, if it is provided by a model, it must be called by the simulator before the Compute routine. Then, your simulator, written for v2.0.0, doesn’t know anything about ChargeCompute(). But, using this mechanism, it can discover that there is something defined by the kim-api that it doesn’t know about but is required by the model...

* A change in the ModelCreate interface from individual calls to set routine pointer (SetComputePointer(), SetDestroyPointer(), etc.) to a unified call for each ModelRoutine: SetRoutinePointer().

OK. Will a ComputeArgumentsDestroy then be needed? My current model does not set any, mainly because I don’t think I need it, but also because I was not able to work out what the prototype should be.

Yes; often this can just be a no-op. (We decided it was simpler overall to require this function to be present than to deal with the complexity of handling two cases of present/absent.)

The necessary prototypes are defined in the KIM_FunctionTypes.hpp header file...

* Two new top-level routines added:

- WriteParameterizedModel(): This provides a mechanism whereby a ModelDriver
  may be asked to write out its current parameters (which may have been
  adjusted from their default values by the simulator's use of SetParameter()
  and ClearThenRefresh()).

So this returns a big string? This makes a lot of sense, since it can be formatted in a more clever way that what the Simulator could do, as the simulator does not know what the parameters mean (nor does it know their “natural” shape, see below).

Not quite, it actually writes them to a specified directory on disk. Since there are, generally, an unknown number of files to be written, this seems like the only workable solution.

- Extension(): This proivides a very general way to extend the KIM API. A
  Simulator and Model must both know about and agree on an extension data
  strucuture. Once this requirement is met, then they may communicate in a
  standard fashion that allows for any task to be performed.

Cool, but a bit dangerous, since it might cause models to lose their generality.

Yes, but we think this is necessary to encourage developers to work with the kim api even if their model’s needs are not fully supported by the standard. (The alternative is to implement their model fully in another system like lammps, and this will make it more difficult down the line to convert to KIM once the api supports the needed features.). This also provides an avenue for experimentation and proposals for official additions to future versions of the api.

May I suggest an optional feature: GetParameterShapeHint(int const parameterIndex) returning a string giving information about the shape of the parameter array. If not supported, it returns the empty string, and the simulator only knows the information returned by GetParameterMetadata(). The list of possible return values can be extended, but includes

“OneDimensional”. A 1-D array, the size is known from GetParameterMetadata().
“Square” An N*N matrix, N can be deducted from the size of the array (N^2).
“Cube” An N*N*N array, the size is N^3.
“UpperTriangular” An upper triangular matrix of size N*N, where the actual size of the array is N*(N-1)/2
“UpperTriangularWithZeros” Like “Square”, but elements below the diagonal are ignored.
“LowerTriangular”
“LowerTriangularWithZeros"
“(2, 43, 32, 10)” Actual dimensions of the array.
“” None of the above, or a scalar.

Interesting suggestion. Can you elaborate on a couple of use cases for this?
Why not just include this info. in the freeform parameter description text? In particular, how can the simulator really make use of this information? How does knowing this, programmatically, at run-time help? If all the program knows is that there is a double UpperTriangular parameter matrix, how is that better than knowing there is a list of parameters of a certain number?

* Some minor new functionality relateing to the KIM API logging mechanism has been added to the KIM::Log class: (PushDefaultVerbosity() and PopDefaultVerbosity()).

Some kind of documentation on how to use the logging would be nice. Also, is there a way for the simulator to discover the errors that the model may have logged, so they may be passed on to the user?

Yes. The next task is to document all the routines in the api and this will give me the chance to describe these features. (This is the last item on the list for final v2 release.)

The examples in the kim-api repo have been updated and adjusted to provide usage examples of the significant additions/changes described above.

Nice! The examples are very useful indeed

Yes, these need to eventually (after v2 official release) be fully redone. I want to carefully think out how best to illustrate all the features and requirements of the api and keep maintenance of the examples manageable...

Ryan

Hi Jakob,

This exists. The tool is called kim-api-v2-collections-info and is in the libexec directory.

You can get the libexec directory from pkg-config:

LIB\_EXEC\_DIR=(pkg-config libkim-api-v2 --variable=libexecdir)

then, for example:

{LIB_EXEC_DIR}/kim-api-v2/kim-api-v2-collections-info models

This prints one row per model with space-separated columns containing:

1) The collection (CWD, user, system)
2) The model name
3) directory where the collection is located
4) the kim-api version string for the kim-api version used to compile the model.

Ryan

Dear Ryan,

[ .... ]

The main need for this is to support backward compatibility in the case where new features are added to the kim-api.

I see, good thinking! So my simulator should contain a list of routines it knows about, and then call IsRoutinePresent() for any routine that it does *not* know about, and then fail gracefully if that is the case.

OK. Will a ComputeArgumentsDestroy then be needed? My current model does not set any, mainly because I don’t think I need it, but also because I was not able to work out what the prototype should be.

Yes; often this can just be a no-op. (We decided it was simpler overall to require this function to be present than to deal with the complexity of handling two cases of present/absent.)

Right now, my simulator calls model->ComputeArgumentsDestroy(&computeargs) but my model does not define a ComputeArgumentsDestroy, and that seems to work. In the future, my model will have to define it (or should my simulator use IsRoutinePresent() to check for it before calling it - just having it seems simpler)?

The necessary prototypes are defined in the KIM_FunctionTypes.hpp header file...

Thanks!

* Two new top-level routines added:

- WriteParameterizedModel(): This provides a mechanism whereby a ModelDriver
may be asked to write out its current parameters (which may have been
adjusted from their default values by the simulator's use of SetParameter()
and ClearThenRefresh()).

So this returns a big string? This makes a lot of sense, since it can be formatted in a more clever way that what the Simulator could do, as the simulator does not know what the parameters mean (nor does it know their “natural” shape, see below).

Not quite, it actually writes them to a specified directory on disk. Since there are, generally, an unknown number of files to be written, this seems like the only workable solution.

Ah yes, I remember that we talked about this feature, it is needed for fitting parameters.

- Extension(): This proivides a very general way to extend the KIM API. A
Simulator and Model must both know about and agree on an extension data
strucuture. Once this requirement is met, then they may communicate in a
standard fashion that allows for any task to be performed.

Cool, but a bit dangerous, since it might cause models to lose their generality.

Yes, but we think this is necessary to encourage developers to work with the kim api even if their model’s needs are not fully supported by the standard. (The alternative is to implement their model fully in another system like lammps, and this will make it more difficult down the line to convert to KIM once the api supports the needed features.). This also provides an avenue for experimentation and proposals for official additions to future versions of the api.

Indeed!

May I suggest an optional feature: GetParameterShapeHint(int const parameterIndex) returning a string giving information about the shape of the parameter array. If not supported, it returns the empty string, and the simulator only knows the information returned by GetParameterMetadata(). The list of possible return values can be extended, but includes

“OneDimensional”. A 1-D array, the size is known from GetParameterMetadata().
“Square” An N*N matrix, N can be deducted from the size of the array (N^2).
“Cube” An N*N*N array, the size is N^3.
“UpperTriangular” An upper triangular matrix of size N*N, where the actual size of the array is N*(N-1)/2
“UpperTriangularWithZeros” Like “Square”, but elements below the diagonal are ignored.
“LowerTriangular”
“LowerTriangularWithZeros"
“(2, 43, 32, 10)” Actual dimensions of the array.
“” None of the above, or a scalar.

Interesting suggestion. Can you elaborate on a couple of use cases for this?
Why not just include this info. in the freeform parameter description text? In particular, how can the simulator really make use of this information? How does knowing this, programmatically, at run-time help? If all the program knows is that there is a double UpperTriangular parameter matrix, how is that better than knowing there is a list of parameters of a certain number?

I guess this may only be useful for a scripting-language simulator like Asap. Right now Asap makes each parameter available as a 1D list of values; this feature would allow it to be reshaped to a more easy-to-use shape. Graphical user interfaces that allow the user to edit parameters might also use it. But I admit this is perhaps too exotic to bother with.

  [ ... ]

Thanks for all your comments!

Jakob

[ .... ]

The main need for this is to support backward compatibility in the case where new features are added to the kim-api.

I see, good thinking! So my simulator should contain a list of routines it knows about, and then call IsRoutinePresent() for any routine that it does *not* know about, and then fail gracefully if that is the case.

Essentially, yes. I've added code to the example simulators that does "the basic test" for this sort of thing. Take a look at that to see how I approached it...

OK. Will a ComputeArgumentsDestroy then be needed? My current model does not set any, mainly because I don’t think I need it, but also because I was not able to work out what the prototype should be.

Yes; often this can just be a no-op. (We decided it was simpler overall to require this function to be present than to deal with the complexity of handling two cases of present/absent.)

Right now, my simulator calls model->ComputeArgumentsDestroy(&computeargs) but my model does not define a ComputeArgumentsDestroy, and that seems to work. In the future, my model will have to define it

Yes.

(or should my simulator use IsRoutinePresent() to check for it before calling it - just having it seems simpler)?

Yes, we decided it was simpler to simply require it. The API will require that a Model provides the ComputeArgumentsDestroy routine. If not, you will get an error from the KIM::Model::Create() routine.

The necessary prototypes are defined in the KIM_FunctionTypes.hpp header file...

Thanks!

I thought it would be helpful :slight_smile:

* Two new top-level routines added:

- WriteParameterizedModel(): This provides a mechanism whereby a ModelDriver
may be asked to write out its current parameters (which may have been
adjusted from their default values by the simulator's use of SetParameter()
and ClearThenRefresh()).

So this returns a big string? This makes a lot of sense, since it can be formatted in a more clever way that what the Simulator could do, as the simulator does not know what the parameters mean (nor does it know their “natural” shape, see below).

Not quite, it actually writes them to a specified directory on disk. Since there are, generally, an unknown number of files to be written, this seems like the only workable solution.

Ah yes, I remember that we talked about this feature, it is needed for fitting parameters.

Right.

- Extension(): This proivides a very general way to extend the KIM API. A
Simulator and Model must both know about and agree on an extension data
strucuture. Once this requirement is met, then they may communicate in a
standard fashion that allows for any task to be performed.

Cool, but a bit dangerous, since it might cause models to lose their generality.

Yes, but we think this is necessary to encourage developers to work with the kim api even if their model’s needs are not fully supported by the standard. (The alternative is to implement their model fully in another system like lammps, and this will make it more difficult down the line to convert to KIM once the api supports the needed features.). This also provides an avenue for experimentation and proposals for official additions to future versions of the api.

Indeed!

May I suggest an optional feature: GetParameterShapeHint(int const parameterIndex) returning a string giving information about the shape of the parameter array. If not supported, it returns the empty string, and the simulator only knows the information returned by GetParameterMetadata(). The list of possible return values can be extended, but includes

“OneDimensional”. A 1-D array, the size is known from GetParameterMetadata().
“Square” An N*N matrix, N can be deducted from the size of the array (N^2).
“Cube” An N*N*N array, the size is N^3.
“UpperTriangular” An upper triangular matrix of size N*N, where the actual size of the array is N*(N-1)/2
“UpperTriangularWithZeros” Like “Square”, but elements below the diagonal are ignored.
“LowerTriangular”
“LowerTriangularWithZeros"
“(2, 43, 32, 10)” Actual dimensions of the array.
“” None of the above, or a scalar.

Interesting suggestion. Can you elaborate on a couple of use cases for this? Why not just include this info. in the freeform parameter description text? In particular, how can the simulator really make use of this information? How does knowing this, programmatically, at run-time help? If all the program knows is that there is a double UpperTriangular parameter matrix, how is that better than knowing there is a list of parameters of a certain number?

I guess this may only be useful for a scripting-language simulator like Asap. Right now Asap makes each parameter available as a 1D list of values; this feature would allow it to be reshaped to a more easy-to-use shape. Graphical user interfaces that allow the user to edit parameters might also use it. But I admit this is perhaps too exotic to bother with.

[ ... ]

OK, yeah, I think we'll leave it to higher-level codes to provide this sort of reshaping feature.

Cheers,

Ryan