First, check that you are using the latest version of Geneious Prime. If you still have this error, check the following recommendation.
Running Muscle 5.1 in Windows may result in this error if you are missing a particular Visual C++ redistributable.
Muscle 5.1 requires vcomp140.dll to be present in your C:\Windows\System32directory.
Either install the Visual C++ Distributable: Download Visual C++ Redistributable for Visual Studio 2015 from Official Microsoft Download Center, or otherwise add the vcomp140.dll to your System32 directory to solve this error.