Merely using GPL-covered software is perfectly fine under any circumstance. The question is whether software that you wrote is also covered by the GPL. This is the case whenever it is a derivative work of GPL-covered software as defined by copyright law, in particular if the software that you distribute is a modified version of GPL-covered software or if your software includes GPL-covered code.
In the description of your software, you have mentioned or implied the following components and relationships:

- you have a repository containing source code for the library and for the tests
- the tests need a GPL library to run
Clearly, the library source code is not a derivative work of the GPL-covered library. Therefore, you cannot be forced to release the library source code or binaries of the library under the GPL. Similarly, the source code of the tests is (likely) not derivative of the GPL-covered library.
However, the runnable tests are probably derivative of the GPL-covered library. Under the terms of the GPL, the source code of your runnable tests must be made available in compliance with the GPL, in particular under a GPL-compatible license. The source code of the tests includes the library's source code.
(You do not actually distribute the runnable tests but only their source code. In my amateur opinion, this difference is not material to this licensing discussion. Assume the runnable tests were licensed in a way so that they could not be distributed under the terms of the GPL. Whether the runnable tests would be distributed (thus committing copyright infringement) or whether you would merely distribute all the parts necessary to create the runnable tests (thus likely committing contributory copyright infringement) – I doubt this would be legal.)
To summarize, this means:
- You are not required to publish your tests.
- If you publish the tests in source form or other form, you must make the source code of the runnable tests under a GPL-compatible license.
- The source code of the tests includes the source code of your main library.
- You can choose different licenses for the libraries and the tests, as long as all licenses are compatible with each other.
- While the license that you choose has to be GPL-compatible, it does not have to be the GPL itself. For example, you could choose the MIT license instead. This should not be a problem for most open source libraries.
There are some niche cases where there would be problems. For example, if you publish the library incl. tests under a license that is not GPL-compatible. Another scenario would be a proprietary software project where you do not intend to share the tests at all, but would later like to license your test suite to a client. This would imply a license of the main library to the client under a GPL-compatible license.
Note that the set of GPL-compatible licenses depends on the exact GPL version. In particular, the Apache 2 license is compatible with GPLv3 but not GPLv2. See the FSF license list for various licenses and the FSF's thoughts on GPL compatibility.