"This problem is with PC architecture - the PCI address range is 32-bit (i.e. 4G of addresses), and each PCI device needs space inside that range. If any memory is present here then most chipsets will map that memory out. Dolphin's cards use a configurable 512M+several smaller address ranges, so a "loss" of up to 1.5G has been seen on systems. There's a way to solve this. With a small cluster (7 or 8 nodes), you do not need a "prefetch-size" of 512M, but rather a smaller range - say 64M. On the machines in question, please do "/opt/scali/sbin/scinode -m". You will have to reboot (possibly powercycle) the node for this to take effect. This will leave you with at least 3.5GB memory remaining. Note, the memory range needed by the card (the prefetch_size) depends on the size of the application that you intend to run (both the number of CPUs and the application buffer sizes), so you may be able torun with as little as 32M, although you will not know that before you test your application." (Dolphin Tech Support)