This month I would like to raise some awareness and discuss options to do LAMMPS development and using LAMMPS on Windows.
LAMMPS is primarily developed on Linux machines by people that are familiar with Linux and other Unix-like operating systems. And for may years that has put some restrictions on compiling and using LAMMPS on Windows machines. Initially, it would only be possible with the Cygwin toolkit, which is some kind of “Unix emulation for Windows”, then we managed to build LAMMPS and package it with a Linux-to-Windows cross-compiler toolchain provided by Fedora. The biggest hurdle was the build system with a dependency on GNU Make, Bourne shell, and Unix shell utilities.
This was improved by adding the option to use CMake as alternate build system, but there were still many details where code needed to be adapted because it used features not available on Windows or that have a different interface in the Windows system libraries. Since the last stable release, however, we have added set of platform abstraction functions to LAMMPS that made it easier to have consistent behavior for certain functionality across all platforms (not just between Linux and Windows). At the same time also Microsoft has significantly improved its Visual Studio Integrated Development Environment and particular the support for Git and CMake.
With the current development branch and thus the upcoming next stable release it should now be possible to compile a large number of packages natively on Windows either from inside Visual Studio 2022 or from the console with externally installed CMake and Git (practically all packages except for those requiring external libraries that do not allow building with CMake or are not ported to Windows).
Compilation with the Intel OneAPI compilers (classic and LLVM) is also possible. The biggest advantage is that this allows compilation of LAMMPS with a much more recent MPI library (i.e. the MSMPI package and SDK version 10.x maintained by Microsoft) instead of the very old package used by the cross-compiled versions since that is the only available MPI package compatible with those compilers.
This should finally put people that want to do LAMMPS development on Windows on a equal footing with the Linux based LAMMPS developers. For the sake of completeness I should also mention that with the availability of the Windows Subsystem for Linux (or other virtual machine software) it also has always been possible to work on Windows, but with significant additional overhead, and that still requires familiarity with the Linux command line and development tools.
I should note that this has benefited substantially by the availability of both Visual Studio and the Intel compilers at no cost to open source developers (and users).
My questions and suggestions for discussion topics for the LAMMPS community are the following:
How important is being able to do software development on Windows for you?
We are thinking about building LAMMPS for Windows installer packages natively on Windows. However, those will have limitations (a rather up-to-date version of Windows 10, fewer packages included). How many of you are bothered by having to install the old MPI package to run the existing Windows binaries?
How many are now encouraged to try build LAMMPS from source on their own instead of using the installer packages?
What functionality or package do those of you that are already doing LAMMPS development on Windows miss the most? Where are you struggling to compile or link your custom binaries?
Thanks in advance for any feedback, comments and suggestions.
I think this could be useful to people and small enterprises willing to do business, as clearly Windows has a bigger user base.
LAMMPS is a key component of materials modelling and has the potential to truly transform our understanding of matter at the molecular level. However, unleashing this potential typically involves very complex simulations and workflows that can only be run on HPC infrastructure, which hardly runs on Windows.
Still, having LAMMPS on Windows can be a good way to attract and train new users. This is a laudable goal, but personally, I wouldn’t spend too much effort on facilitating software development on Windows. BTW, what is the fraction of LAMMPS code that has been developed on Windows?
I don’t think that is the right question to ask. Instead, the question should be: what is the fraction of code that is going to be developed on Windows.
For a person predominantly using Linux, it is easy to dismiss Windows because of its unfamiliar and in some ways “clumsy” command line interface and because of the many problems and quirks historically associated with Windows. This applies to me as well. I am far more comfortable on Linux with rather old-fashioned tools that have a steep earning curve for most people.
However, the matter of the fact is, that Windows has continually improved over the years and become more developer friendly. For example, I don’t think there is an easier way to get started on LAMMPS development than installing the community edition of Visual Studio 2022. “Everything” is included when you select the option to do development of command line applications, so you just need to install visual studio, then use its Git interface to clone the LAMMPS source code and confirm that the main CMake file is in the “cmake” folder. From there on it just takes a couple of clicks and one or two selections on drop down lists until you have a working LAMMPS executable with most features enabled and testing included. I always had and still have a dislike for IDEs, but putting that aside there is no denying how well this all works now and I cannot think of an alternative to get started programming LAMMPS with less effort. On a Linux system in comparison it can take a while to find all the necessary packages or you need to install a large bundle with lots of items you don’t need or will ever use.
In comparison, at least in my personal impression, the mainstream Linux user interface with the Gnome shell is going backwards. If it wasn’t for alternatives like XFCE and LXDE, I would be in trouble since it disagrees with how I prefer to operate (same as the MacOS interface, BTW, which it reminds me of by quite a bit). Also, Ubuntu’s push toward use of snap packages for core parts of its software stack is extremely annoying and costing performance.
On top of that, one has to acknowledge that workstations today can be incredibly powerful. I do a lot of development and testing on a meager Intel NUC with an i5-10210U CPU that has a nominal clock of 1.6GHz. Yet with 4 cores and turbo boost frequencies of up to 4.1GHz (only dropping to ~3.2GHz for sustained re-compilations), in combination with a fast SSD and ccache, compilation and running tests is quite fast. I rarely need to move to bigger larger machines. So people that have a high-end workstation with, say dual socket 32-core CPUs (and possibly a couple of high-end GPUs) have an incredible amount of compute power available and those can be operated with Windows just as well as with Linux.
I agree with “let’s not be snobbish about Linux”. But I feel like most people who want to work on LAMMPS development would also be deploying LAMMPS on clusters, and they’d have to wrangle the Linux command line to work on clusters anyway.
For myself I find that I’ve had a pretty good run doing LAMMPS development even though I’m on a Windows box by using WSL. This way I get to use Windows for my day-to-day work while doing all my scientific computational work on Linux. I’ve even run production runs for GROMACS on WSL and can’t see a significant performance drop from the WSL layer – the latest versions of WSL even (claim to) interface with GPUs for accelerated computation. VS Code has WSL plugins that allow you to code in a Windows-side IDE (with a WSL Bash prompt in a subwindow if desired). These days Windows even packs a native X11 / Wayland window manager, and you can natively move files back and forth between Windows and WSL filesystems using either the WSL command line or Windows Explorer.
If we wanted to improve the accessibility of LAMMPS usage in Windows environments, I think getting a package working in the Conda ecosystem would be very effective, especially if we could then get LAMMPS running on cloud GPUs via Google Colab with Jupyter notebook frontends, like with the Making It Rain project (GitHub - pablo-arantes/making-it-rain: Cloud-based molecular simulations for everyone). One can dream …
It primarily needs somebody that knows how to do this. I think everything else in LAMMPS is ready for that. There are just minor tweaks needed occasionally here and there (e.g. to avoid issues with file- and path-names containing spaces). My personal experience with Conda is not so good. I have found the quality of the packaging vary substantially between good and poor. But that was on Linux.
I think I didn’t mention it explicitly in my original post, but if you install Git and CMake separately you can build LAMMPS from the command line just as well. Depending on which compilers and build tools you prefer (I prefer Ninja-build over MSBuild), you may need to use the batch environment command line sessions provided and installed by the respective compiler vendors so the compilers can be found by CMake. We’ve been running a GitHub Action on Windows for LAMMPS to compile it with the MSVC compilers and run the unit tests for several months now: Actions · lammps/lammps · GitHub
(Tip: Install the “App Installer” from the Microsoft store, and then you have the “winget” command which acts like a package manager and allows to install and update applications with similar convenience than Linux package managers offer and without the complexity and overhead of “external” tools like chocolatey).
It’s been said before, but it’s worth mentioning again that many people may be forced to use Windows by their employer or school, whatever their familiarity with Linux may be.
Many organizations maintain a fleet of desktop computers running either Windows or macOS: then they probably run some Linux servers, but Linux on workstations or laptops is mostly an afterthought in many places. Moreover, even when a scientist is reasonably skilled to install Linux on their own, they may not be allowed to do so by their organization’s IT department. This usually comes out of maintenance concerns, i.e. they are simply not equipped to handle a Linux computer on their network. For the same reason, that organization may also restrict the usage of WSL or virtual machines, leaving folks out of option.
With that in mind, I would think it most important to make sure that a scientific app like LAMMPS could be (1) installed reasonably easily by a local user with admin access (like a personal PC), but also that (2) it can be installed in an automated/scripted fashion from a centrally-approved self-service app repository that the organization maintains. I can share privately the name of the tool used for this purpose where I work.
Related to the above, the main reason why one would use a local Windows machine rather than a remote cluster is for local visualization. Like, for plotting software or more complex tools like OVITO or VMD. @stree rightfully mentioned Anaconda, but I’m not sure where it stands with respects to that latter kind of apps. Even not counting potential shared features like fix imd, just communicating with the developers of those tools to try and make the installation procedures more consistent would be worth it.
Giacomo
P.S. I forgot to include on the first draft, but with Apple consciously making macOS a less and less flexible OS, Windows could well become soon the best alternative to Linux for science.
The precompiled Windows installer packages use the NullSoft scripted installer tool. That should allow to do an automated installation with default settings (or customized options?) from the command line. That feature is implicitly used to uninstall an existing package in case installation is tried a second time, for example with a newer version. So that might be of use and I should probably add some information about this to the package website.
In addition, LAMMPS is available in the Microsoft Store (App) for even simpler installation (but due to restrictions to packaging such Apps without MPI and most auxiliary tools and documentation.
The Microsoft Store App is indeed very useful for personal PCs, but it is often blocked by administrators in a larger org. If the NullSoft installer is sufficiently established to work easily with what those admins use as alternative, then it may be sufficient.
FWIW, I just created and submitted a manifest file for winget.
I tested it locally and it does install the LAMMPS package with just one minor issue. Because it is not signed with a code signing certificate, I had to tell Windows to install it anyway. So I suspect that for proper support of automated package installation I would need to get a code signing certificate.
I regularly use MDAnalysis for post-processing, and while I’ve not personally used Py3Dmol.js I’ve read very good things about it. Both live in the conda-forge ecosystem.
So those packages can (begin to) meet users’ needs for post-processing and viz. We just need the LAMMPS engine now …
FWIW, the latest version of LAMMPS is now also available via winget:
C:\Users\akohlmey>winget install lammps.lammps
Found LAMMPS -- MD Simulation Software [LAMMPS.LAMMPS] Version 2Jun2022
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://rpm.lammps.org/windows/64bit/LAMMPS-64bit-2Jun2022.exe
██████████████████████████████ 110 MB / 110 MB
Successfully verified installer hash
Starting package install...
Successfully installed
The following should install LAMMPS from the command line with default settings: LAMMPS-64bit-2Jun2022.exe /S
If you want to modify the destination folder then: LAMMPS-64bit-2Jun2022.exe /S /D=X:\Path\To\Install\Dir