it has been over a year since I started a discussion on how to improve the relation between LAMMPS users and the core developers and how to get more people involved in the project. This was particularly focused on having more people participate in discussions in the forum and having more people helping with code maintenance and improving the documentation.
Looking back, I think we can say that at least the situation with the forum has improved markedly and to the better. There are now several people that regularly respond to questions and offer their help, and sometimes also contribute to more general discussions. We are very grateful for that and appreciate this a lot. More recently, I have even started to wonder whether this has been too successful, since there seems to be a tendency toward questions that are only peripherally linked to LAMMPS.
Since then, things have also changed in the core LAMMPS developer team. People have moved on to a new job with new responsibilities, or have retired from their job. Both led to having less time available to do the regular maintenance tasks that come with managing a software project as big as LAMMPS. This is particularly challenging, since the number of contributions submitted as pull requests is still steady and quite large (just look at the last feature release) so the load of processing these, is now effectively distributed over fewer shoulders. At the same time, there are still ongoing refactoring and maintenance projects with lots of work still unfinished that are aimed at modernizing LAMMPS and making it more consistent, more accessible, and more maintainable.
So my question is again: what can we do? Especially, what can we do that does not require us to do (much) more work (since we are already stretched thin)?
Or perhaps I should ask this differently: what are the reasons that people are not contributing more? What are barriers that could be lowered? Or what needs to be communicated better?
I think a better question to ask would be “What kind of contributions do the core LAMMPS developers expect from the community?”. I know there are volunteer needed threads on Github. However, you said yourself that you were stretched thin so I would think that just increasing the number of PRs would make it worse.
One minor thing that is unclear for me is what should I do with small contributions. Let’s say I found some typos on a single manual page. In order to make a correction, I should pull the latest LAMMPS code, create a new branch, fix the typos, and fill a PR. This takes a lot of time (especially for someone who never used git), compared to a few seconds required to make the actual change in code. Additionally, it feels excessive to create a separate PR to just correct a few words.
I think a solution here would be to create a place somewhere (Github or a pinned thread on this forum) where people could describe such small changes. Someone (not necessarily one of the core programmers) would gather all of them and incorporate them, let’s say once per month.
Creating such a thread here would have an additional benefit that people not proficient with programming (or without Github account) could also contribute something.
Ultimately, we need people that are interested in contributing to LAMMPS by helping with the tasks that are currently shouldered almost exclusively by the core developers. Participating in the discussions of the forum is one item. As I have mentioned, there have been significant improvements compared to the mailing list and the initial situation when we switched to the forum. But more important is help with the processing of tasks on GitHub, and ongoing refactoring projects. The ultimate goal would be that people become sufficiently experienced with these tasks, that we would invite them to become part of the core developer group.
Not necessarily. There is a difference between pull requests that add completely new features (and are sometimes created by people that are unaware of the conventions and preferences) and pull requests that improve the quality and maintainability of the core code and existing features. One needs practice to get a sense of how to assess the code of others and how to change it to be closer to the revised design goals. This applies to both, the code itself and the documentation. Specifically in the parts of the documentation aimed at developers, there are lots of holes and room for improvement. Contributing developer documentation would be a good chance for multiple people to learn more about LAMMPS and its implementation itself, because if someone can correctly summarize and describe how programming should be done, then this demonstrates the capability to help with code review.
Since this is going to be a benefit in the long run, there is certainly the willingness to invest some time into guiding people through this. Expanding the testing facilities is another area where there are plenty of holes.
This was the intention behind the “Code Clinic” event last summer. Unfortunately, due to circumstances beyond my control, there was not much time to prepare and announce it, so participation was limited and perhaps also participants expected something different than what was actually offered. If there is interest, we can do another edition with more lead time and more prepared material. I am also open to try out something different. I think the important point would be that there is a “net benefit” for the LAMMPS developers. In other words, we are willing to invest effort into training people and sharing our experience and expectations if this will lead to help with maintaining LAMMPS, but we don’t want to end up training people that are only looking to become able to or better at doing their research (i.e. what they should be trained on by their adviser) and that have no intention to contribute effort keep LAMMPS going (or have the intention only at the very beginning and then are caught up in the realities of doing research, this has been observed multiple times).
I agree. There are two options. 1) I (or others) usually have a “collected small changes” pull request open (currently Collected small fixes by akohlmey · Pull Request #3649 · lammps/lammps · GitHub), where you can just add a comment with the change you would like to add (make the change locally, do git diff and then copy the output). 2) You can create an issue with the same kind of content.
Assisting with how to use git in the way it is used by the LAMMPS developers is certainly something where we can offer some help. Since development is deeply connected with using git, the desire to learn and practice effective use of git (and GitHub) is certainly a prerequiste for participation in the LAMMPS maintenance.
Thanks for a detailed answer, I think it should be converted to a separate pinned topic on this forum (and somewhere on Github), so more people can see it. I agree with @srtee that the unfulfilled goals of the LAMMPS Code Clinic should be posted on Github. I would also add there the planned refactoring projects which aren’t currently assigned to someone. Possibly also with a difficulty/complexity level.
I think this would help people, who want to contribute, to find suitable tasks. It would definitely help me, although I can only pitch in a few hours per week.
It is my observation that feature request posts on GitHub have very little impact. We have a whole lot of them already. Some of the topic of the Code Clinic are already posted as feature request issues (albeit not with the same level of detail).
My post and question is aimed at something different. Getting features implemented, that we have identified as desirable, is very nice and helpful. And it can be something that people can look at if they are looking for a task to work on. If what they are looking for is just a project with sufficient “hack value™”.
What I am trying (and apparently failing) to understand, goes beyond that. We are looking for people that want to do more than just implement features or refactor or document or test the code. We need people that also want to be involved in identifying and outlining such projects, people that want to be part of the LAMMPS development process and want to have a voice when decisions are made. We need people that want to be part of making LAMMPS a better software in general.
For example, reviewing pull requests and assisting contributors to improve their submission usually does not require much programming on your own. Also, one does not have to be an expert in all of LAMMPS; different people can review and assist with different parts of a pull request. It regularly happens that multiple developers get involved with integrating a contribution.
So any information that makes us understand what would motivate people to get involved in code or documentation maintenance or what we can do to integrate those that are already somewhat involved more. Similarly, I would like to better understand what are the biggest barriers that keep people away from getting more closely involved, and what we could do to lower them.
Or to put it very bluntly: we need to find and integrate people into the team that are ready to take over when the current LAMMPS developers will retire or otherwise no longer available. If we wait until (too many of) those things happen, it will be too late.
Speaking for myself, what you are looking for has only become clear in your last few posts in this thread. I, like @mkanski , thought you were looking for volunteers to help with short-term contributions to improve particular areas or aspects of the code. Instead, it seems like you want dedicated team members to help out the core dev team long-term, starting with smaller contributions and assisting those who want to contribute, with a path to work more closely with the team and the possibility to become a full developer.
If that’s the case, my questions are simply “Where do I sign up?” and “How soon can I start?”
Which just illustrates why I am asking for feedback (I know I am not very good at communicating this because to me this is self-evident. I feel a bit like a blind man asked to describe colors).
Please send me a message with your email address to my email address.
Next week?
I would propose to first arrange a Zoom call to go over some basics, intentions, plans, projects, preferences etc. When you send me an email, you can propose some dates and time windows.
I think LAMMPS’s user base (or at least potential-dev-base) needs to know the roadmaps for LAMMPS’s:
goals (what is LAMMPS? why is it not GROMACS or OpenMM? what’s next on the to-do list?)
contributors (how do I go from occasional bugfixes to ?? to being part of the core dev team?)
sustainability (who will pay for LAMMPS? how can I get LAMMPS dev work recognised as part of my job description / grant deliverables?)
governance (who will get to decide all of the above, and will it be done with diversity, equity and inclusion in mind?)
It would certainly help to have a community manager helping to actively communicate all these to the userbase and collecting continuous feedback.
I don’t think the unclarity (or less-explicit-ness) of this information is determinative of people not contributing (I blame it on the general academic problem with supporting research software engineers), but it would certainly help to make this information explicit and highlighted, especially when envisioning any kind of core dev team transition.
I’d certainly be happy to help – with the caveat being my Australian (Brisbane) time zone.
For the people working at Sandia, LAMMPS is an asset, since they hire staff that help researchers to realize their projects with LAMMPS. So for them LAMMPS has to be accessible, sufficiently flexible and adaptable. For others LAMMPS is what you make of it.
Basically you can look at 1.3. LAMMPS features — LAMMPS documentation and say the goals are “keep this and add more of it”. There are features in LAMMPS that nobody has anticipated.
Mostly see above. I would say goals are the goals of the individual contributors, for as long as they don’t conflict with the needs listed above. For the core team, it may be summarized by: a) improve portability, accessibility. better performance, and make it easier to extend the code. b) improve support for accelerated platforms through Kokkos, c) improve programming interfaces, reduce redundancy, improve consistency, remove clutter, more testing d) improve documentation, tutorials etc.
I think the main path is by making suggestions and - if met with positive feedback - implementing them. This will automatically start more detailed discussions and at some point there will be emails or comments addressed to you with questions. If you demonstrate that you can do certain things well, you will be asked to do more of it and get involved in workshops and similar. That is how it worked for me.
This is very tricky business. For as long as Sandia sees LAMMPS as an important asset, there will be some staff time contributed to maintaining it. It can also happen that they offer a subcontract (e.g. some of my time is paid for by a subcontract that Temple has for a project run by Sandia to provide developer and user support), but money for those is limited.
My personal conviction is that, if LAMMPS continues to grow as it currently does, at some point this is not sufficient and alternate means of funding need to be found. My favorite approach in that regard would be to form a non-profit organization that would become tax-exempt and then collect donations (from individual people or research groups but also from corporate sponsors) and use those to fund full time staff or part time work from researchers.
There also could be a “bounty system” for adding features.
This is rather democratic. For the most part we discuss until there is a consensus. Rarely, questions are put to a vote. Steve as the original and principal author reserves the right to veto changes that he absolutely disagrees with (has not happened as far as I recall).
Altogether, this is rather ad hoc and there are no fixed explicit rules.
If somebody would show up with a desire to do something like that, it would be welcome. It is not something that any of the current core developers is eager to do. We are better at programming and source code is the language in which we communicate best.
Overall, the main attitude is: if somebody wants to contribute something that is useful, does not interfere with the rest of LAMMPS and works as advertised, it can be integrated. In the past this was done without much regard for code quality, consistency, portability etc. As LAMMPS has grown, this has become a problem and thus there are now (documented!) preferences, conventions, and requirements, tests and the GitHub code review process. That has helped to avoid many of the problems of the past that only became visible after integration into the code base. My personal goal is, that people will consider LAMMPS as a “professional grade” software, i.e. not like the project of a first year graduate student that has never programmed in C++ before. Or put differently, bug should be eliminated faster than they are added.
At least for the forum, this has been more of a benefit than a disadvantage. With people scattered around the globe and not just concentrated in the US, there is a better “quality of service” as there would always be somebody available to respond. Add to this that many computer geeks already maintain unusual work schedules.
Sidebar: being on the other side of the globe should be less of a problem than my development experience, when the public git repository was synchronized with the (otherwise not accessible) Sandia subversion repository for LAMMPS. This meant that whatever patches I had sent via email to Sandia and that were committed to the repository, would only surface in my upstream on the next day, and in combination with new commits directly from folks at Sandia. Sometimes there were conflicts. Since most of the time, those changes were published immediately as patches. I was often on the losing side of playing catch-up to fix all the issues that changes to the core code required in the OPENMP (then USER-OMP) package. Sometimes, there were periods where the package was broken for multiple patch releases in a row. The 24 hour interval was brutal. Whenever I would travel to Europe, I would feel better, because it gave me an additional 6 hours to detect the need for and to implement fixes.
Thanks, having all this explicitly spelled out is really helpful!
So BUKSCMFL then (Benevolent United-Kingdom-Style Constitutional Monarch For Life).
My personal suspicions are always that when a situation seems unsustainable, it almost certainly already is, and when a situation is clearly unsustainable it is often too late to fix without radical changes. I wonder if OMSF would be suitable for providing the non-profit administrative services that would be needed for long-term sustainability.
Thanks for further clarification. I would also be interested, but as I said, I only can work on LAMMPS a few hours each week. Which leads to one more question: How much time would you (and other core members) expect these new programmers to spend on LAMMPS (let’s say, per month)?
There is no fixed expectation. Any constructive and useful contribution is welcome. For all members of the LAMMPS team, the available time can vary substantially between weeks or months depending on what other, non-LAMMPS commitments we have.
I would say the only time related expectations are to be responsive (say, within a business day or two) when there are questions or requests for clarifications, and to not take on projects that are too ambitious for the time available (so that branches or pull requests would sit in Draft mode for a long time). There are lots of things that can be done in small(er) increments. The same goes for non-programming tasks like discussions/review/advice on pull requests or similar.
