IANAL/IANYL. That said, it seems to me that what you're doing is writing a proprietary plugin for a GPL program. It's a piece of proprietary code designed to slot into a GPL program, that may or may not get loaded at run-time. We can ignore the bit about the shim library being MIT-licensed on its own, because it's part of the work as distributed so must come under the GPL that covers the main work.
The FSF have already given their opinion on this practice. They say that
If the main program and the plugins are a single combined program then this means you must license the plug-in under the GPL or a GPL-compatible free software license and distribute it with source code in a GPL-compliant way.
Whether your plugin and the main program would form a single combined program depends, they say, on how tightly they communicate; that is, on the nature of the interface between main program and plugin. If you
establish intimate communication by sharing complex data structures, or shipping complex data structures back and forth, that can make them one single combined program
You haven't shown us your code, and in any case I'm not qualified to analyse it. But you should take a long, hard look at your plugin library, and be honest with yourself about how tightly it couples to the main program. Unless communication across the interface is "arm's-length", what you propose to do may well violate the GPL.