Tests are not generally part of a program, so I would consider a license that requires tests to be published to be non-free. Conversely, I think that deliberately withholding tests for open-source software is technically fine, but quite uncooperative.
Unlike the actual source code or build scripts, access to tests is not required to inspect or modify a software. In practice, lack of tests is a barrier to successfully forking a project as the fork might become more buggy than the original project. However, this barrier doesn't actually prevent succesful forking. The relevant software freedoms can still be exercised without tests.
A notable project that doesn't publish its test suite is SQLite, but it uses a very “cathedral” development process (“open source, not open contribution”).