Can I distribute an installer script which installs X, P and Q on the user's computer when they run it?
I have the feeling that you are treading on thin ice here and you should definitely consult a lawyer with experience in open-source licensing before betting a company on it.
Your safest option is to distribute X without any plugins and let the user ...
According to the GPL FAQ:
I would like to bundle GPLed software with some sort of installation software. Does that installer need to have a GPL-compatible license? (#GPLCompatInstaller)
No. The installer and the files it installs are separate works. As a result, the terms of the GPL do not apply to the installation software.
Judging by your last paragraph, the backend plugins are loaded as libraries into the application that contains your library. The GPL license considers that the application and all involved libraries are a single work and that the GPL terms and conditions must apply to every part of that work.
As a consequence, if you distribute your library build with/...